🤖🧠 Concerto: How Joint 2D-3D Self-Supervised Learning Is Redefining Spatial Intelligence

🗓️ 09 Nov 2025
📚 AI News & Trends

The world of artificial intelligence is rapidly evolving and self-supervised learning has become a driving force behind breakthroughs in computer vision and 3D scene understanding. Traditional supervised learning relies heavily on labeled datasets which are expensive and time-consuming to produce. Self-supervised learning, on the other hand, extracts meaningful patterns without manual labels allowing models to ...

#SelfSupervisedLearning #ComputerVision #3DSceneUnderstanding #SpatialIntelligence #AIResearch #DeepLearning

✨Scaling Spatial Intelligence with Multimodal Foundation Models

📝 Summary:
SenseNova-SI is a new scaled multimodal foundation model that achieves superior spatial intelligence. By using 8 million diverse data samples, it sets unprecedented performance on various spatial benchmarks. The models are publicly released to foster further research.

🔹 Publication Date: Published on Nov 17

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.13719
• PDF: https://arxiv.org/pdf/2511.13719
• Project Page: https://huggingface.co/sensenova/SenseNova-SI-1.1-InternVL3-8B
• Github: https://github.com/OpenSenseNova/SenseNova-SI

🔹 Models citing this paper:
• https://huggingface.co/sensenova/SenseNova-SI-InternVL3-8B
• https://huggingface.co/sensenova/SenseNova-SI-InternVL3-2B
• https://huggingface.co/sensenova/SenseNova-SI-1.1-InternVL3-2B

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MultimodalAI #FoundationModels #SpatialIntelligence #ComputerVision #AI

✨COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence

📝 Summary:
COOPER is a unified MLLM that integrates depth and segmentation modalities to enhance spatial intelligence. It uses adaptive interleaved reasoning, improving spatial reasoning by 6.91%. Learning to generate auxiliary modalities also strengthens spatial understanding, boosting distance and size es...

🔹 Publication Date: Published on Dec 4

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.04563
• PDF: https://arxiv.org/pdf/2512.04563
• Github: https://github.com/zhangzef/COOPER

🔹 Models citing this paper:
• https://huggingface.co/Starrrrrry/COOPER-AMG
• https://huggingface.co/Starrrrrry/COOPER

✨ Datasets citing this paper:
• https://huggingface.co/datasets/Starrrrrry/COOPER_Train_Set

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MLLM #SpatialIntelligence #ComputerVision #AI #DeepLearning

🔥 Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

💡 The paper presents JoyAI-Image, a unified multimodal foundation model that integrates visual understanding, text-to-image generation, and instruction-guided image editing. The model combines a spatially enhanced Multimodal Large Language Model with a Multimodal Diffusion Transformer, allowing for a shared multimodal interface between perception and generation. The authors propose a scalable training recipe that incorporates unified instruction tuning, long-text rendering supervision, spatially grounded data, and general and spatial editing signals. This design enables the model to achieve broad multimodal capabilities while strengthening geometry-aware reasoning and controllable visual synthesis. The experiments demonstrate that JoyAI-Image achieves state-of-the-art or highly competitive performance across various benchmarks, including understanding, generation, long-text rendering, and editing tasks. The model's bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables it to move beyond general visual competence toward stronger spatial intelligence. The results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models. Overall, the paper contributes to the development of a unified multimodal model that can effectively understand and generate visual content with enhanced spatial intelligence.

📅 Published on May 5

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.04128
• PDF: https://arxiv.org/pdf/2605.04128
• GitHub: https://github.com/jd-opensource/JoyAI-Image ⭐ 2.1k

🤖 Models citing this paper:
• https://huggingface.co/jdopensource/JoyAI-Image-Edit

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/stevengrove/JoyAI-Image-Edit-Space
• https://huggingface.co/spaces/stevengrove/JoyAI-Image-Edit
• https://huggingface.co/spaces/Merlinus001/JoyAI-Image-Edit-Space

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #UnifiedFoundationModels #MultimodalDiffusionTransformers #SpatialIntelligence #MultimodalGeneration

🔥 OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs

💡 The paper introduces OVO-S-Bench, a comprehensive benchmark for evaluating the ability of multimodal language models to understand spatial information from continuous video streams. The problem addressed is that existing benchmarks for spatial intelligence either evaluate models on full videos or focus on events rather than spatial structure, and do not account for the need to reason about places and layouts from partial information.

To address this, the authors created a benchmark consisting of 1680 human-annotated questions spanning 348 source videos, with each question having a query timestamp and an evidence interval. The questions cover four levels of abstraction, from basic perception to complex spatial reasoning and mapping. The annotation process involved 12 trained annotators who also served as cross-reviewers, ensuring high quality through multiple rounds of review.

The results show that even the best performing model, Gemini-3.1-Pro, trails human experts by 27 points, with the most challenging task being allocentric mapping. Interestingly, models that are specifically fine-tuned for streaming and spatial tasks actually perform worse than their original backbones, suggesting that these models may not be effectively using the spatial information in the video streams. The authors also found that using chain-of-thought reasoning can amplify spatial errors when the model is not grounded in the stream.

Overall, the OVO-S-Bench benchmark provides a challenging testbed for evaluating and improving the spatial intelligence of multimodal language models, and highlights the need for further research in this area to address the limitations of current models.

📅 Published on Jun 2

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.03890
• PDF: https://arxiv.org/pdf/2606.03890
• Project Page: https://internlm.github.io/OVO-S-Bench/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLanguageModels #SpatialIntelligence #StreamingVideoAnalysis #VideoUnderstandingBenchmarks #MultimodalLLMEvaluation