🔖 ImageBind: One Embedding Space To Bind Them All

📝 This project is a significant step forward in understanding and connecting information from diverse sources like images, text, audio, video, and even motion sensor data.

⚙️ Supports 6 Modalities:

📷 Image
📝 Text
🔈 Audi
🎥 Video
🦴 IMU sensor data (e.g., accelerometer)
🙄 Depth/Thermal & 3D data
Interestingly, only some modalities had labels, yet ImageBind learned to align them through self-supervised learning.


💫 Key Features:

..No need for paired data (e.g., images and audio don’t have to be aligned)..Leverages contrastive learning for learning joint embedding space
..Competes with CLIP and AudioCLIP, but with better accuracy and coverage..Enables zero-shot retrieval (e.g., finding relevant video using just a sentence)


📌 Repo: https://github.com/facebookresearch/ImageBind

🔍 By: https://xn--r1a.website/DataScienceN 🌟

#ImageBind #MultimodalAI #MetaAI #DeepLearning #SelfSupervised

🐈‍⬛ TTT Long Video Generation 🐈‍⬛

▶️ A novel architecture for video generation, adapting the #CogVideoX 5B model by incorporating #TestTimeTraining (TTT) layers.
Adding TTT layers into a pre-trained Transformer enables generating a one-minute clip from text storyboards.
Videos, code & annotations released 💙

🔗 Review: https://t.ly/mhlTN
📄 Paper: arxiv.org/pdf/2504.05298
🌐 Project: test-time-training.github.io/video-dit
🧑‍💻 Repo: github.com/test-time-training/ttt-video-dit

#AI #VideoGeneration #MachineLearning #DeepLearning #Transformers #TTT #GenerativeAI

🔍 By: https://xn--r1a.website/DataScienceN5

🚀 CoMotion: Concurrent Multi-person 3D Motion 🚶‍♂️🚶‍♀️

Introducing CoMotion, a project that detects and tracks detailed 3D poses of multiple people using a single monocular camera stream. This system maintains temporally coherent predictions in crowded scenes filled with difficult poses and occlusions, enabling online tracking through frames with high accuracy.

🔍 Key Features:
- Precise detection and tracking in crowded scenes
- Temporal coherence even with occlusions
- High accuracy in tracking multiple people over time

🎁 Access the code and weights here:
🔗 Code & Weights 
🔗 View Project

This project advances 3D human motion tracking by offering faster and more accurate tracking of multiple individuals compared to existing systems.

#AI #DeepLearning #3DTracking #ComputerVision #PoseEstimation

🎙 By: https://xn--r1a.website/DataScienceN