AI & ML Papers
34.3K subscribers
7.45K photos
657 videos
24 files
8.27K links
Advancing research in Machine Learning – practical insights, tools, and techniques for researchers.

Admin: @HusseinSheikho || @Hussein_Sheikho
Download Telegram
AI & ML Papers
Photo
🔥 Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

💡 The paper presents JoyAI-Image, a unified multimodal foundation model that integrates visual understanding, text-to-image generation, and instruction-guided image editing. The model combines a spatially enhanced Multimodal Large Language Model with a Multimodal Diffusion Transformer, allowing for a shared multimodal interface between perception and generation. The authors propose a scalable training recipe that incorporates unified instruction tuning, long-text rendering supervision, spatially grounded data, and general and spatial editing signals. This design enables the model to achieve broad multimodal capabilities while strengthening geometry-aware reasoning and controllable visual synthesis. The experiments demonstrate that JoyAI-Image achieves state-of-the-art or highly competitive performance across various benchmarks, including understanding, generation, long-text rendering, and editing tasks. The model's bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables it to move beyond general visual competence toward stronger spatial intelligence. The results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models. Overall, the paper contributes to the development of a unified multimodal model that can effectively understand and generate visual content with enhanced spatial intelligence.


📅 Published on May 5

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.04128
• PDF: https://arxiv.org/pdf/2605.04128
• GitHub: https://github.com/jd-opensource/JoyAI-Image ⭐ 2.1k

🤖 Models citing this paper:
• https://huggingface.co/jdopensource/JoyAI-Image-Edit

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/stevengrove/JoyAI-Image-Edit-Space
• https://huggingface.co/spaces/stevengrove/JoyAI-Image-Edit
• https://huggingface.co/spaces/Merlinus001/JoyAI-Image-Edit-Space

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #UnifiedFoundationModels #MultimodalDiffusionTransformers #SpatialIntelligence #MultimodalGeneration
❤3
AI & ML Papers
Photo
🔥 SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

💡 The paper introduces SenseNova-U1, a unified multimodal model that integrates understanding and generation into a single process, overcoming the traditional divide between these two tasks. Current large vision-language models treat understanding and generation as separate problems, leading to fragmented architectures and misaligned representation spaces. The authors argue that this divide hinders the emergence of native multimodal intelligence and propose a new paradigm, NEO-unify, which views understanding and generation as synergistic aspects of a single process.

The authors present two variants of SenseNova-U1, built on dense and mixture-of-experts understanding baselines, and demonstrate their performance across various tasks, including text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. The models also excel in image synthesis, infographic generation, and interleaved vision-language generation, showing strong semantic consistency and visual fidelity.

The paper provides detailed information on model design, data preprocessing, pre- and post-training, and inference strategies, supporting community research. The results show that SenseNova-U1 models perform strongly in vision-language-action and world model scenarios, indicating a broader roadmap where models can think and act across modalities in a native manner. The authors conclude that multimodal AI should focus on building a unified system, rather than connecting separate systems, allowing necessary capabilities to emerge from within. Overall, the paper contributes to the development of unified multimodal models that can integrate understanding and generation, paving the way for more advanced and native multimodal intelligence.


📅 Published on May 12

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.12500
• PDF: https://arxiv.org/pdf/2605.12500
• GitHub: https://github.com/OpenSenseNova/SenseNova-U1 ⭐ 1.6k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #NEOunifyArchitecture #VisionLanguageModels #MultimodalGeneration #UnifiedIntelligenceModels
AI & ML Papers
Photo
🔥 Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

💡 The paper introduces Boogu-Image-0.1, an open-source unified multimodal understanding and generation model family that delivers competitive performance in high-quality text-to-image generation, fast inference, instruction-based editing, and bilingual text rendering. The model family consists of Base, Turbo, Edit, and Edit-Turbo variants.

The problem addressed in the paper is that closed-source multimodal systems achieve strong performance through system-level integration rather than a single model, yet their internal practices remain largely undisclosed. The authors demonstrate that targeted improvements in model understanding, data quality, and training pipelines, coupled with genetic inference-time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets.

The method used to achieve this involves making improvements in model understanding, data quality, and training pipelines. The authors also use genetic inference-time scaling to enhance performance. The model is trained on a dataset of 208.62 million unique images, and the theoretical training cost is approximately 400,000 dollars.

The results show that Boogu-Image-0.1 consistently matches or surpasses other open-source models across standard benchmarks and achieves results approaching leading closed-source systems. The authors share practical discussions and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified multimodal understanding and generation. The code is available on GitHub.

Overall, the paper contributes to the development of open-source multimodal models that can achieve competitive performance with closed-source systems, and provides a valuable resource for the research community.


📅 Published on Jul 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.13125
• PDF: https://arxiv.org/pdf/2607.13125
• Project Page: https://boogu.org/

🤖 Models citing this paper:
• https://huggingface.co/Boogu/Boogu-Image-0.1-Edit
• https://huggingface.co/Boogu/Boogu-Image-0.1-Base
• https://huggingface.co/Boogu/Boogu-Image-0.1-Turbo

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/RioShiina/ImageGen
• https://huggingface.co/spaces/multimodalart/Boogu-Image
• https://huggingface.co/spaces/Tinchote/ImageGen

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #TextToImageGeneration #OpenSourceAI #MultimodalGeneration #InstructionBasedEditing
AI & ML Papers
Photo
🔥 TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training

💡 The paper presents TorchUMM, a unified codebase for evaluating and analyzing multimodal models that can understand, generate, and edit across visual and textual modalities. The development of unified multimodal models has been hindered by the diversity of model architectures and training paradigms, making it challenging to compare and analyze different models. To address this issue, TorchUMM provides a comprehensive framework that supports a wide range of models and tasks, including multimodal understanding, generation, and editing. The codebase includes a broad spectrum of models and integrates both established and novel datasets to evaluate perception, reasoning, compositionality, and instruction-following abilities. By providing a unified interface and standardized evaluation protocols, TorchUMM enables fair and reproducible comparisons across heterogeneous models, facilitating the development of more capable unified multimodal systems. The codebase is designed to foster deeper insights into the strengths and limitations of different models, and its availability is expected to accelerate the development of unified multimodal models. The TorchUMM codebase is available for use, allowing researchers to evaluate and analyze their models in a standardized and reproducible manner. Overall, the paper contributes to the development of unified multimodal models by providing a unified framework for evaluation, analysis, and post-training, which can help to advance the field of multimodal modeling.


📅 Published on Apr 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2604.10784
• PDF: https://arxiv.org/pdf/2604.10784
• Project Page: https://aifrontierlab.github.io/TorchUMM/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalModeling #UnifiedCodebase #MultimodalUnderstanding #VisualTextualModalities #MultimodalGeneration
❤1