Digital Therapeutics Startups in Mental Health – Investors Mapping Landscape 2Q 2024
This mapping highlights the relationships between mental health DTx startups and their investors, showcasing a diverse range of venture capital firms supporting innovation in digital therapeutics for mental health.

This workbook is a comprehensive guide aimed at helping project teams ensure the safety, reliability, and ethical governance of AI systems.

1️⃣ It provides a structured approach to integrating AI safety into the development and deployment of AI technologies, particularly within the public sector.

2️⃣ Key components of AI safety include performance, which ensures models are accurate and performant with metrics like accuracy, precision, and recall. Reliability involves maintaining consistent performance under varying conditions. Security focuses on protecting AI systems from adversarial attacks and data breaches. Robustness ensures systems operate reliably under unforeseen conditions.

3️⃣ AI Safety Self-Assessment involves conducting safety self-assessments at each project stage, identifying and mitigating risks specific to the AI project's context, and documenting actions in a Risk Management Plan.

4️⃣ Activities for AI Safety include practical exercises to build understanding and application of AI safety concepts, as well as case studies and group-based activities to contextualize AI safety in real-world scenarios.

5️⃣ Engagement and training are supported through facilitator and participant versions of the workbook to aid workshops, along with resources for continuous learning and feedback integration.

6️⃣ Stakeholder involvement emphasizes participatory approaches for comprehensive safety assessments and engaging stakeholders to understand the impacts of AI systems and improve safety protocols.

This is one of the coolest ideas for scaling synthetic data

Proposes 1 billion diverse personas to facilitate the creation of diverse synthetic data for different scenarios.

It's easy to generate synthetic data but hard to scale up its diversity which is essential for its application.

This paper proposes a novel persona-driven data synthesis methodology to generate diverse and distinct data covering a wide range of perspectives.

Previous works synthesize data using either instance-driven approaches (e.g., using seed corpus) or key-point-driven methods (e.g., using topic/subject). Both of these approaches lack the desired coverage, quality, and perspectives needed to robustly scale the data synthesis process.

To measure the quality of the synthetic datasets, they performed an out-of-distribution evaluation on MATH. A fine-tuned model on their synthesized 1.07M math problems achieves 64.9% on MATH, matching the performance of gpt-4-turbo-preview at only a 7B scale.

Their method is not only effective for MATH problems, but it can also be used to generate logical reasoning problems, instructions, game NPCs, tool development, knowledge-rich text, and many more use cases.

AI Agents That Matter

Performs a careful analysis of existing benchmarks, analyzing across additional axes like cost, proposes new baselines

1. AI agent evaluations must be cost-controlled
2. Jointly optimizing accuracy and cost can yield better agent design
3. Model developers and downstream developers have distinct benchmarking needs
4. Agent benchmarks enable shortcuts
5. Agent evaluations lack standardization and reproducibility

New AI market map for Accounting by a16z

Anthropic CEO Dario Amodei says another generation or two of scale is needed before the error rate is low enough to have autonomous AI agents.

❗️Researchers prerinted a 497 reference review introducing and reviewing language models and agents.

They focus to chemistry/biochem on language models (80!), but cover 24 language model agents across science.

Paper.
Code.

This is amazing, Kyutai leading the charge on real time voice assistants and as a true open-science non-profit, will release the code and details.

Moshi by Kyutai just owned the stage!

You can play around with the demo directly here.

Architecture
1. 7B Multimodal LM (speech in, speech out)
2. 2 channel I/O - Streaming LM constantly generates text tokens as well as audio codecs (tunable)
3. Achieves 160ms latency (with a Real-Time Factor of 2)
4. The base text language model is a 7B (trained from scratch) - Helium 7B
5. Helium 7B is then jointly trained on w/ text and audio codecs
6. Speech codec is based on a Mimi (their inhouse audio compression model)
7. Mimi is a VQ-VAE capable of 300x compression factor - trained on both semantic and acoustic information
8. Text to Speech Engine supports 70 different emotions and styles like whispering, accents, personas, etc

Training/ RLHF
1. The model is fine-tuned on 100K transcripts generated by Helium itself.
2. These transcripts are highly detailed, heavily annotated with emotion and style, and conversational.
3. Text to Speech Engine is further fine-tuned on 20 hours of audio recorded by Alice and licensed.
4. The model can be fine-tuned with less than 30 minutes of audio.
5. Safety: Generated audio is watermarked (possibly w/ audioseal) & generated audios are indexed in a database
6. Trained on Scaleway cluster of 1000 H100 GPUs

Inference
1. The deployed demo model is capable of bs=2 at 24GB VRAM (hosted on Scaleway and Hugging Face)
2. Model is capable of 4-bit and 8-bit quantisation
3. Works across backends - CUDA, Metal, CPU
4. Inference code optimised with Rust
5. Further savings to be made with better KV Caching, prompt caching, etc.

Future plans
1. Short-term technical report and open model releases.
2. Open model releases would include the inference codebase, the 7B model, the audio codec and the full optimised stack.
3. Scale the model/ refine based on feedback except Moshi 1.1, 1.2, 2.0
4. License as permissive as they can be (yet to be decided)

Just 8 team members put all of this together! 🔥

After using it IRL, it feels magical to have such a quick response. It opens so many avenues: research assistance, brainstorming/Steelman discussion points, language learning, and more importantly, it's on-device with the flexibility to use it however you want!

This is an interesting article from Sequoia which argues the tech industry needs $600B in AI revenue to justify the money spent on GPUs and data centers.

OpenAI is the biggest AI pure play and is at $3.4B annual run rate. This feels like a bubble unless products worth buying show up.

There is no doubt that there will be a lot of money made from AI. The question is whether it will be enough to support a $3 trillion valuation for Nvidia?

This is why replacing knowledge workers is the play. Anything else isn’t big enough.

ElevenLabs announced a new ‘Iconic Voices’ feature

It allows users to have text read by AI-generated voices of Hollywood stars

Most notably, the Company reached licensing agreements for the AI voices of Judy Garland, James Dean, Burt Reynolds, and more.

A groundbreaking study suggests that depression and anxiety are brain circuit function disorders.

The researchers classified six distinct biotype profiles for humans diagnosed with anxiety or depression disorders, indicating there are multiple specific neural pathways that can cause depression and anxiety within humans.

These discoveries are important within a clinical context, as they can help predict how mental health patients will react to different treatments based on their biotype.

Synthesia launched Synthesia 2.0, the world’s first video communications platform built for businesses

- New features like AI screen recorder and interactivity
- Improved creation & translation workflows
- Next-gen AI avatars (soon with hands!).

Next-gen AI avatars - now with hands.

Full-body, fully controllable AI avatars that can interact with their environment and gesticulate with their hands — coming to Synthesia later this year.

Personal Avatars - 5 min to create and high-quality 🗣️

Use a webcam, phone or camera to record at home. Keep your natural background if you like.

They come with amazing lip-sync and natural voice, together with the ability to replicate a person’s voice in over 30 languages.

AI Screen Recorder - 10x faster, fully editable and choose your voice or one of Synthesia.

A new AI screen recorder perfectly transcribes the voiceover, syncs the screen capture with it, and adds automatic zoom effects to highlight key actions.

AI Video Assistant - bulk video creation🔗 + brand kits

Soon, users will be able to simply select a branded template, provide a link to their knowledge center, and the AI video assistant will transform all the articles into short, engaging videos.

Interactive videos🎮

Create rich video experiences with features such as clickable hotspots, embedded forms, quizzes, and personalized CTAs for an entirely new video experience.

Updated translation flow + multilingual video player 🌐

Also introducing 1-click translations to all language variants of same video, and a new, responsive type of video player.

Once a video is shared, player will automatically play it in the viewer's language.

SoftBank’s $10 Billion-Plus Plan to Get Into the AI Race Centers on Power and Chips

SoftBank has recently talked with banks about borrowing money to fund investment of up to $10 billion in energy-related projects, where AI is driving an enormous increase in demand.

SoftBank also is exploring ways to gain access to a large volume of Nvidia’s graphics processing units, which are critical for AI development.

SoftBank is less focused on generative AI startup investments and has mostly sat out the wave of startup fundraisings in the past year or so.

CEO Masayoshi Son even blocked SoftBank’s Vision Fund from participating in an investment in Mistral, an open-source model developer based in Paris, because he felt it could jeopardize SoftBank's relationship with OpenAI.

The episode signaled to the Vision Fund team that it should not pursue investments in large-language models that could compete with OpenAI.

SoftBank’s growing interest in energy, including potential breakthroughs in areas such as solar and nuclear technologies, reflects a growing view that securing enough power for data centers is becoming just as difficult as getting enough AI chips.

The company’s big spending in power and chips would go hand in hand with its investments in data centers as well as AI startups in Silicon Valley and elsewhere that develop models and applications. Last month, SoftBank announced plans to build a large-scale AI data center in Osaka, Japan, using a major part of the land and buildings formerly occupied by electronics maker Sharp’s factory, which recently ceased operations.
Son has said repeatedly that the centerpiece of his AI ambitions is Arm, the U.K. chip designer SoftBank acquired in 2016.

SoftBank is expecting that Arm, which licenses its chip architecture design to virtually all major tech companies, could play a key role in the evolution of AI chips in the future.

Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning

Building on our previous works on Neural Developmental Programs (NDPs), researchers propose a class of self-organizing neural networks capable of synaptic and structural plasticity.

Biological neural networks grow and develop over a lifetime of learning, and so should artificial neural networks! Here, they showed that a ‘self-assembling’ network can grow, and learn from experiences in different environments starting from randomly connected or empty networks.

Microsoft's new research project, GraphRAG, explores the use of knowledge graphs and LLMs for enhanced RAG.

RAG is an important part of most LLM-based tools and the majority use vector similarity as the search technique, which means

- they struggle to connect related information to provide new insights (such as answer questions that require gathering information from multiple sources in a dataset)

-  they have difficulty understanding complex concepts in large documents or collections.

GraphRAG is open source.

Researchers at UCSD and MIT introduced Open-TeleVision

It's an open-source tele-op system that allows users to control robots from thousands of miles away.

𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧 lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘢𝘵𝘢𝘳.

It's also accessible from any device with a web browser, including VR.

Google DeepMind researchers published new research introducing JEST.

It's a new method that accelerates AI model training while significantly reducing computing requirements.

Faster training capabilities = the acceleration of advanced model releases is just getting started

The race for the first AI-discovered medication to successfully reach patients is on

With new and improved Generative AI models for drug discovery and development emerging in the past six months, the race is becoming far more competitive.
 
Currently, eleven AI-discovered drugs are in phase 2 clinical trials worldwide. It will be interesting to see who will be the first company to reach phase 3.

Powerful LLMs trained on biochemical data released in the past six months are now changing this landscape by allowing the whole biotech community to enter the race.

So why is Generative AI more powerful in drug discovery and development compared to previous AI models?

There are four main reasons:

𝟏. 𝐆𝐞𝐧𝐀𝐈 𝐰𝐨𝐫𝐤𝐬 𝐰𝐞𝐥𝐥 𝐞𝐯𝐞𝐧 𝐨𝐧 𝐬𝐦𝐚𝐥𝐥 𝐝𝐚𝐭𝐚𝐬𝐞𝐭𝐬. Because it is pre-trained on vast amounts of data, it doesn't need a lot of data to fine-tune a model for a specific task.

𝟐. 𝐆𝐞𝐧𝐀𝐈 𝐢𝐬 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 (𝐎𝐛𝐯𝐢𝐨𝐮𝐬𝐥𝐲?!). These models are trained generatively and can be used to generate de-novo biochemical data, resulting in better-performing biological products.

𝟑. 𝐆𝐞𝐧𝐀𝐈 𝐨𝐮𝐭𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐬 𝐩𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧𝐬 in most cases compared to previous AI paradigms used on biochemical data.

 𝟒. 𝐆𝐞𝐧𝐀𝐈 𝐢𝐬 𝐄𝐱𝐩𝐥𝐚𝐢𝐧𝐚𝐛𝐥𝐞. Unlike previous AI models for biochemical data, which were black boxes, GenAI can be explained using the built-in attention mechanism.

For these reasons, using Generative AI allows companies to generate better-performing medications and to reduce the time to market in their discovery pipeline dramatically.

AI for Health published a report focused on the main applications of #generativeAI in #healthcare.

To maximize genAI's impact in healthcare, strong legal frameworks and innovative privacy tools are crucial.

Addressing AI #explainability, ensuring #data accessibility, and #training healthcare professionals are essential steps.

Poe launched ‘Previews’, a new feature allowing users to generate and interact with web apps directly in chats

(Basically Claude Artifacts, but you can do it with any model)

According to the startup, Previews works best using LLMs that excel in coding.

Flashcards.

Drum machine

Promising news from #Google in the domain of generative pharma

“Developing #therapeutics is a lengthy and expensive process that requires the satisfaction of many different criteria, and #AI models capable of expediting the process would be invaluable.

However, the majority of current AI approaches address only a narrowly defined set of tasks, often circumscribed within a particular domain.

To bridge this gap, we introduce Tx-LLM, a generalist LLM fine-tuned from PaLM-2 which encodes knowledge about diverse therapeutic modalities."

#pharma #nextpharma #healthcare #artificialintelligence #google