Pleias published the largest dataset to date with automated OCR correction, 1 billion words in English, French, German and Italian.
OCR quality is primary concern of digitization in any large scale organization. Scans are not always on well-preserved and in many case existing OCR tools are not able to properly parse specific fonts or formats, especially in other languages than English.
Automated post-OCR correction has been made possible thanks to progress in open LLM research and several months of dedicated training and alignment by Pleias.
Results are now encouraging most of the time, on a variety of European languages, even when the text is severely degraded.
OCR quality is primary concern of digitization in any large scale organization. Scans are not always on well-preserved and in many case existing OCR tools are not able to properly parse specific fonts or formats, especially in other languages than English.
Automated post-OCR correction has been made possible thanks to progress in open LLM research and several months of dedicated training and alignment by Pleias.
Results are now encouraging most of the time, on a variety of European languages, even when the text is severely degraded.
Google announced Med-Gemini, a family of Gemini models fine-tuned for medical tasks
Achieves SOTA on 10 of the 14 benchmarks, spanning text, multimodal & long-context applications.
Surpasses GPT-4 on all benchmarks!
Achieves SOTA on 10 of the 14 benchmarks, spanning text, multimodal & long-context applications.
Surpasses GPT-4 on all benchmarks!
arXiv.org
Capabilities of Gemini Models in Medicine
Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of complex...
Meta announced Better & Faster Large Language Models via Multi-token Prediction
Large language models such as GPT and Llama are trained with a next-token prediction loss.
Large language models such as GPT and Llama are trained with a next-token prediction loss.
huggingface.co
Paper page - Better & Faster Large Language Models via Multi-token Prediction
Join the discussion on this paper page
Another triumph for Self-Play. Self-Play Preference Optimization (SPPO) has surpassed (iterative) DPO, IPO, Self-Rewarding LMs, and others on AlpacaEval, MT-Bench, and the Open LLM Leaderboard.
Remarkably, Mistral-7B-instruct-v0.2 fine-tuned by SPPO achieves superior performance to GPT-4 0613 without relying on any GPT-4 responses.
Remarkably, Mistral-7B-instruct-v0.2 fine-tuned by SPPO achieves superior performance to GPT-4 0613 without relying on any GPT-4 responses.
All you need is Kolmogorov–Arnold Network (KAN)
Kolmogorov-Arnold network obliterates Deepmind's results with much smaller networks and much more automation.
KANs also discovered new formulas for signature and discovered new relations of knot invariants in unsupervised ways.
GitHub.
Kolmogorov-Arnold network obliterates Deepmind's results with much smaller networks and much more automation.
KANs also discovered new formulas for signature and discovered new relations of knot invariants in unsupervised ways.
GitHub.
arXiv.org
KAN: Kolmogorov-Arnold Networks
Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation...
OpenAI is about to go after Google search.
This could be the most serious threat Google has ever faced.
OpenAI's SSL certificate logs now show they created search.chatgpt.com
Microsoft Bing would allegedly power the service.
This shouldn’t be too surprising, considering:
1. OpenAI has a web crawler, GPTBot.
2. ChatGPT Plus users can also use Browse with Bing to search the web.
3. Microsoft Bing uses OpenAI’s GPT-4, customized for search.
This could be the most serious threat Google has ever faced.
OpenAI's SSL certificate logs now show they created search.chatgpt.com
Microsoft Bing would allegedly power the service.
This shouldn’t be too surprising, considering:
1. OpenAI has a web crawler, GPTBot.
2. ChatGPT Plus users can also use Browse with Bing to search the web.
3. Microsoft Bing uses OpenAI’s GPT-4, customized for search.
Academic benchmarks are losing their potency. There’re 3 types of LLM evaluations that matter:
1. Privately held test set but publicly reported scores, by a trusted 3rd party who doesn’t have their own LLM to promote.Scale’s latest GSM1k is a great example. They are an unbiased neutral party who ensures that the test data is not leaked into anyone’s training.
2. Public, comparative benchmarks like Lmsys.org Chatbot Arena, reported in ELO score. You can’t game democracy.
3. Privately curated, internal benchmarks for each company’s own use cases. You can’t game your customers.
1. Privately held test set but publicly reported scores, by a trusted 3rd party who doesn’t have their own LLM to promote.Scale’s latest GSM1k is a great example. They are an unbiased neutral party who ensures that the test data is not leaked into anyone’s training.
2. Public, comparative benchmarks like Lmsys.org Chatbot Arena, reported in ELO score. You can’t game democracy.
3. Privately curated, internal benchmarks for each company’s own use cases. You can’t game your customers.
H-GAP is a generalist model for humanoid control.
Trained on large MoCap-derived data, it can generate diverse, natural motions & transfer skills to new tasks without fine-tuning!
Paper.
Trained on large MoCap-derived data, it can generate diverse, natural motions & transfer skills to new tasks without fine-tuning!
Paper.
Yingchen Xu
Humanoid Control with a Generalist Planner
Meta and Georgia institute of technology released a dataset + SOTA AI models to help accelerate research on Direct Air Capture — a key technology to combat climate change.
OpenDAC23 is the largest dataset of Metal Organic Frameworks characterized by their ability to adsorb CO2 in the presence of water — an order of magnitude larger than any other pre-existing dataset at this precision.
OpenDAC23 is the largest dataset of Metal Organic Frameworks characterized by their ability to adsorb CO2 in the presence of water — an order of magnitude larger than any other pre-existing dataset at this precision.
⚡4
"Neuro-GPT: Towards A Foundation Model for EEG" is available
Code on GitHub.
Pre-trained model on HuggingFace.
Code on GitHub.
Pre-trained model on HuggingFace.
arXiv.org
Neuro-GPT: Towards A Foundation Model for EEG
To handle the scarcity and heterogeneity of electroencephalography (EEG) data for Brain-Computer Interface (BCI) tasks, and to harness the power of large publicly available data sets, we propose...
⚡4
Unlearn AI released a new neural network architecture for learning to create digital twins of patients.
⚡4
The future of AI language models may lie in predicting beyond the next word: Multi-Token Prediction
Studies suggest that the human brain predicts multiple words at once when understanding language, utilizing both semantic and syntactic information for broader predictions - now researchers from Meta are hoping to train their LLMs to do the same.
The authors proposed a new training method for language models, called "multi-token prediction," which predicts multiple words simultaneously instead of just the next word.
"Our 13B parameter models solves 12% more problems on HumanEval [benchmark test] and 17% more on MBPP than comparable next-token models."
Predicting hierarchical representations of future input and generating a multi-token response enhances performance, coherence, and reasoning capabilities (particularly for larger models) as it attempts to mimic the human brain.
Works particularly well with coding, and may become a key feature of advanced language models to be released later this year that include the number "5".
Studies suggest that the human brain predicts multiple words at once when understanding language, utilizing both semantic and syntactic information for broader predictions - now researchers from Meta are hoping to train their LLMs to do the same.
The authors proposed a new training method for language models, called "multi-token prediction," which predicts multiple words simultaneously instead of just the next word.
"Our 13B parameter models solves 12% more problems on HumanEval [benchmark test] and 17% more on MBPP than comparable next-token models."
Predicting hierarchical representations of future input and generating a multi-token response enhances performance, coherence, and reasoning capabilities (particularly for larger models) as it attempts to mimic the human brain.
Works particularly well with coding, and may become a key feature of advanced language models to be released later this year that include the number "5".
Intel introduced the biggest neuromorphic computer in the world - Hala Point.
This system, commissioned by Sandia National Labs, integrates 1152 Intel Labs Loihi 2 chips in a three-dimensional array.
Changes to Loihi from gen one to gen two mean this computer can be used to run spiking neural networks and optimization problems as well as converted mainstream deep learning models at excellent power efficiency.
This system, commissioned by Sandia National Labs, integrates 1152 Intel Labs Loihi 2 chips in a three-dimensional array.
Changes to Loihi from gen one to gen two mean this computer can be used to run spiking neural networks and optimization problems as well as converted mainstream deep learning models at excellent power efficiency.
EE Times
World’s Biggest GPU, plus Software-First AI Chip Design with SiFive’s Chris Lattner - EE Times Podcast
On this podcast: GPU giants battle it out, plus a conversation with SiFive’s Chris Lattner on developing AI chips software-first.
The latest version of the Unity game engine is now available to developers as a preview.
Unity 6 Preview includes new features to make XR development easier, including Composition Layers which can significantly increase the quality of text, UI, photos, and videos in XR.
Unity 6 Preview includes new features to make XR development easier, including Composition Layers which can significantly increase the quality of text, UI, photos, and videos in XR.
Road to VR
New Unity 6 Feature Makes it Easier for XR Devs to Maximize Text & UI Quality
The latest version of the Unity game engine—which is the basis for the vast majority of XR games made today—is now available to developers as a preview. Unity 6 Preview includes new features to make XR development easier, including ‘Composition Layers’ which…
Gen_AI_in_Global_Financial_Institutions_1714984764.pdf
2.3 MB
BCG has released an whitepaper on'Transformation’s Edge: The State of GenAI in Global Financial Institutions.'
Decision makers at some of the world’s leading financial institutions (FIs) believe Generative AI (GenAI) presents a transformative business opportunity, according to new survey from BCG.
But only a few have made significant progress in pursuing that vision - for instance, by establishing delivery teams or developing detailed plans for use cases.
Indeed, many FI leaders say more groundwork is needed to assemble the tools and capabilities that would foster a winning GenAI proposition. Acquiring specialized talent is a key priority.
Where FIs are using GenAI in practice, it is most often to serve support functions such as call center services or software development, rather than transform business-critical operations at scale.
Key Survey Insights
1. 85% of financial institutions in BCG's survey believe GenAI will be highly disruptive or transformational.
2. But only 2% have a fully developed GenAI talent strategy.
3. Just 26% are actively investing a significant proportion of their innovation budgets in GenAI implementation.
• Almost three quarters of survey respondents are in the early stages of use case development.
• The most progressed GenAI use cases focus on boosting internal productivity, rather than re-shaping critical functions or inventing new business models.
Decision makers at some of the world’s leading financial institutions (FIs) believe Generative AI (GenAI) presents a transformative business opportunity, according to new survey from BCG.
But only a few have made significant progress in pursuing that vision - for instance, by establishing delivery teams or developing detailed plans for use cases.
Indeed, many FI leaders say more groundwork is needed to assemble the tools and capabilities that would foster a winning GenAI proposition. Acquiring specialized talent is a key priority.
Where FIs are using GenAI in practice, it is most often to serve support functions such as call center services or software development, rather than transform business-critical operations at scale.
Key Survey Insights
1. 85% of financial institutions in BCG's survey believe GenAI will be highly disruptive or transformational.
2. But only 2% have a fully developed GenAI talent strategy.
3. Just 26% are actively investing a significant proportion of their innovation budgets in GenAI implementation.
• Almost three quarters of survey respondents are in the early stages of use case development.
• The most progressed GenAI use cases focus on boosting internal productivity, rather than re-shaping critical functions or inventing new business models.
Visa and Allium Labs: less than 10% of stablecoin transactions are "organic payments activity", which excludes activities from bots and large-scale traders.
PayPal, Stripe are among fintechs expanding in stablecoins.
PayPal, Stripe are among fintechs expanding in stablecoins.
Bloomberg.com
More Than 90% of Stablecoin Transactions Aren’t From Real Users, Study Finds
More than 90% of stablecoin transaction volumes aren’t coming from genuine users, according to a new metric co-developed by Visa Inc., suggesting such crypto tokens may be far away from becoming a commonly used means of payment.
LLMs are better than humans at designing reward functions for robotics
New genetic atlas of brain development
Researchers created an atlas detailing the early genetic development of the brain from weeks 6 to 13 of embryonic growth.
The atlas is expected to aid in understanding the developmental errors leading to pediatric brain tumors and could help in developing targeted treatments.
Additionally, the study is part of the broader “Human Developmental Cell Atlas” project, which aims to map the genetic development of various organs.
Key Facts:
1. Extensive Brain Mapping: The atlas provides a detailed map of gene activation and cellular development in the brain during early embryonic stages.
2. Potential Clinical Applications: The insights from this atlas are being used to study and understand the origins of brain cancer in children, potentially leading to new therapeutic strategies.
3. Collaborative Research Effort: This research is part of a larger effort, funded by prominent foundations, to create comprehensive cellular atlases for multiple organs, enhancing our understanding of human development and disease.
Researchers created an atlas detailing the early genetic development of the brain from weeks 6 to 13 of embryonic growth.
The atlas is expected to aid in understanding the developmental errors leading to pediatric brain tumors and could help in developing targeted treatments.
Additionally, the study is part of the broader “Human Developmental Cell Atlas” project, which aims to map the genetic development of various organs.
Key Facts:
1. Extensive Brain Mapping: The atlas provides a detailed map of gene activation and cellular development in the brain during early embryonic stages.
2. Potential Clinical Applications: The insights from this atlas are being used to study and understand the origins of brain cancer in children, potentially leading to new therapeutic strategies.
3. Collaborative Research Effort: This research is part of a larger effort, funded by prominent foundations, to create comprehensive cellular atlases for multiple organs, enhancing our understanding of human development and disease.
news.ki.se
Early genetic development of the brain mapped
In an article published in Nature, researchers from Karolinska Institutet present an atlas of the early development of the brain. The atlas can be used, among other things, to find out what went wrong in the development of brain tumors in children and also…
Stack Overflow announced partnership with OpenAI
As part of this collaboration:
1. OpenAI will utilize Stack Overflow’s OverflowAPI product and collaborate with Stack Overflow to improve model performance for developers who use their products.
2. Stack Overflow will utilize OpenAI models as part of their development of OverflowAI and work with OpenAI to leverage insights from internal testing to maximize the performance of OpenAI models.
As part of this collaboration:
1. OpenAI will utilize Stack Overflow’s OverflowAPI product and collaborate with Stack Overflow to improve model performance for developers who use their products.
2. Stack Overflow will utilize OpenAI models as part of their development of OverflowAI and work with OpenAI to leverage insights from internal testing to maximize the performance of OpenAI models.
Wayve raised $1.05 billion from SoftBank, Microsoft and NVIDIA.
Wayve
Wayve Raises Over $1 Billion Led by SoftBank to Develop Embodied AI Products for Automated Driving
Discover how Wayve is revolutionizing autonomous mobility with embodied intelligence. $1.05 billion Series C investment from SoftBank, NVIDIA, and Microsoft.