AI & ML Papers
34.3K subscribers
7.46K photos
657 videos
24 files
8.27K links
Advancing research in Machine Learning – practical insights, tools, and techniques for researchers.

Admin: @HusseinSheikho || @Hussein_Sheikho
Download Telegram
AI & ML Papers
Photo
🔥 Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

💡 The paper presents JoyAI-Image, a unified multimodal foundation model that integrates visual understanding, text-to-image generation, and instruction-guided image editing. The model combines a spatially enhanced Multimodal Large Language Model with a Multimodal Diffusion Transformer, allowing for a shared multimodal interface between perception and generation. The authors propose a scalable training recipe that incorporates unified instruction tuning, long-text rendering supervision, spatially grounded data, and general and spatial editing signals. This design enables the model to achieve broad multimodal capabilities while strengthening geometry-aware reasoning and controllable visual synthesis. The experiments demonstrate that JoyAI-Image achieves state-of-the-art or highly competitive performance across various benchmarks, including understanding, generation, long-text rendering, and editing tasks. The model's bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables it to move beyond general visual competence toward stronger spatial intelligence. The results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models. Overall, the paper contributes to the development of a unified multimodal model that can effectively understand and generate visual content with enhanced spatial intelligence.


📅 Published on May 5

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.04128
• PDF: https://arxiv.org/pdf/2605.04128
• GitHub: https://github.com/jd-opensource/JoyAI-Image ⭐ 2.1k

🤖 Models citing this paper:
• https://huggingface.co/jdopensource/JoyAI-Image-Edit

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/stevengrove/JoyAI-Image-Edit-Space
• https://huggingface.co/spaces/stevengrove/JoyAI-Image-Edit
• https://huggingface.co/spaces/Merlinus001/JoyAI-Image-Edit-Space

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #UnifiedFoundationModels #MultimodalDiffusionTransformers #SpatialIntelligence #MultimodalGeneration
❤3
AI & ML Papers
Photo
🔥 SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

💡 The paper introduces SenseNova-U1, a unified multimodal model that integrates understanding and generation into a single process, overcoming the traditional divide between these two tasks. Current large vision-language models treat understanding and generation as separate problems, leading to fragmented architectures and misaligned representation spaces. The authors argue that this divide hinders the emergence of native multimodal intelligence and propose a new paradigm, NEO-unify, which views understanding and generation as synergistic aspects of a single process.

The authors present two variants of SenseNova-U1, built on dense and mixture-of-experts understanding baselines, and demonstrate their performance across various tasks, including text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. The models also excel in image synthesis, infographic generation, and interleaved vision-language generation, showing strong semantic consistency and visual fidelity.

The paper provides detailed information on model design, data preprocessing, pre- and post-training, and inference strategies, supporting community research. The results show that SenseNova-U1 models perform strongly in vision-language-action and world model scenarios, indicating a broader roadmap where models can think and act across modalities in a native manner. The authors conclude that multimodal AI should focus on building a unified system, rather than connecting separate systems, allowing necessary capabilities to emerge from within. Overall, the paper contributes to the development of unified multimodal models that can integrate understanding and generation, paving the way for more advanced and native multimodal intelligence.


📅 Published on May 12

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.12500
• PDF: https://arxiv.org/pdf/2605.12500
• GitHub: https://github.com/OpenSenseNova/SenseNova-U1 ⭐ 1.6k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #NEOunifyArchitecture #VisionLanguageModels #MultimodalGeneration #UnifiedIntelligenceModels
AI & ML Papers
Photo
🔥 AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Reward

💡 The paper introduces AlphaGRPO, a novel framework that enhances multimodal generation capabilities in unified multimodal models. The problem addressed is the need for improved multimodal generation without requiring an additional cold-start stage. To solve this, the authors apply Group Relative Policy Optimization to AR-Diffusion Unified Multimodal Models, enabling self-reflective refinement and decompositional verifiable reward mechanisms.

The method involves using Decompositional Verifiable Reward, which decomposes complex user requests into atomic, verifiable semantic and quality questions. These questions are then evaluated by a general multimodal language model to provide reliable and interpretable feedback. This approach allows the model to perform advanced reasoning tasks, including reasoning text-to-image generation and self-reflective refinement.

The results show that AlphaGRPO yields robust improvements across multimodal generation benchmarks, including GenEval, TIIF-Bench, DPG-Bench, and WISE. The framework also achieves significant gains in editing tasks on GEdit without training on editing tasks. The experiments demonstrate that the self-reflective reinforcement approach effectively leverages inherent understanding to guide high-fidelity generation, validating the effectiveness of AlphaGRPO. Overall, the paper contributes to the development of more advanced and reliable multimodal generation models.


📅 Published on May 12

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.12495
• PDF: https://arxiv.org/pdf/2605.12495
• Project Page: https://huangrh99.github.io/AlphaGRPO/
• GitHub: https://github.com/huangrh99/AlphaGRPO ⭐ 37

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalGeneration #UnifiedMultimodalModels #SelfReflectiveLearning #DecompositionalReward #MultimodalDeepLearning
🔥 Cosmos 3: Omnimodal World Models for Physical AI

💡 The paper introduces Cosmos 3, an omnimodal world model that can process and generate multiple data types, including language, image, video, audio, and action sequences, through a unified mixture-of-transformers architecture. This model is designed to jointly process and generate different modalities, effectively combining vision-language models, video generators, world simulators, and world-action models into a single framework. The Cosmos 3 model achieves state-of-the-art performance in various understanding and generation tasks, demonstrating its ability to serve as a scalable and general-purpose backbone for embodied agents.

The key contribution of the paper is the development of a unified architecture that can handle multiple input-output configurations, allowing for seamless integration of different modalities. The model is evaluated on a diverse suite of tasks, including text-to-image and image-to-video generation, and policy modeling, and achieves superior performance compared to existing models. The authors also make their code, model checkpoints, curated synthetic datasets, and evaluation benchmark available under an open-source license to accelerate research and deployment in Physical AI.

The results show that the Cosmos 3 model establishes a new state-of-the-art in various tasks, and its post-trained models were ranked as the best open-source models in text-to-image and image-to-video generation, as well as policy modeling. The availability of the code and model checkpoints is expected to facilitate further research and development in Physical AI, and the Cosmos 3 model has the potential to become a widely-used backbone for embodied agents. Overall, the paper presents a significant contribution to the field of Physical AI, demonstrating the effectiveness of omnimodal world models in achieving state-of-the-art performance in a wide range of tasks.


📅 Published on Jun 1

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.02800
• PDF: https://arxiv.org/pdf/2606.02800
• Project Page: https://research.nvidia.com/labs/cosmos-lab/cosmos3/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#OmnimodalLearning #MultimodalGeneration #PhysicalAI #WorldModels #EmbodiedIntelligence
AI & ML Papers
Photo
🔥 Vision as Unified Multimodal Generation

💡 The paper introduces a unified multimodal model that formulates computer vision tasks as generation problems using natural language and visual prompts. This approach allows for a single model to perform a wide range of vision tasks without requiring task-specific architectures. The model, called SenseNova-Vision, uses natural-language instructions and optional visual prompts to specify tasks and generates responses as text, images, or mixed text-and-image outputs. To support large-scale training, the authors created the SenseNova-Vision Corpus, a computer-vision instruction-response corpus that spans text, image, and mixed targets. The model is trained on this corpus, along with auxiliary multimodal data, and achieves performance comparable to specialized systems across diverse vision tasks, including detection, OCR, keypoint estimation, segmentation, and camera pose estimation. The results demonstrate that a single unified model can match leading task-specialized systems, suggesting that unified multimodal generation is a scalable route for integrating computer vision capabilities into general-purpose foundation models. The model and corpus are publicly available, providing a valuable resource for the research community. Overall, the paper presents a significant contribution to the field of computer vision, offering a unified and flexible approach to tackling a wide range of vision tasks.


📅 Published on Jul 7

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.06560
• PDF: https://arxiv.org/pdf/2607.06560

🤖 Models citing this paper:
• https://huggingface.co/sensenova/SenseNova-Vision-7B-MoT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/sensenova/SenseNova-Vision-Corpus-50M
• https://huggingface.co/datasets/sensenova/SenseNova-Vision-Benchmark

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalGeneration #VisionTasks #NaturalLanguageProcessing #ComputerVision #MultimodalLearning
AI & ML Papers
Photo
🔥 Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

💡 The paper introduces Boogu-Image-0.1, an open-source unified multimodal understanding and generation model family that delivers competitive performance in high-quality text-to-image generation, fast inference, instruction-based editing, and bilingual text rendering. The model family consists of Base, Turbo, Edit, and Edit-Turbo variants.

The problem addressed in the paper is that closed-source multimodal systems achieve strong performance through system-level integration rather than a single model, yet their internal practices remain largely undisclosed. The authors demonstrate that targeted improvements in model understanding, data quality, and training pipelines, coupled with genetic inference-time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets.

The method used to achieve this involves making improvements in model understanding, data quality, and training pipelines. The authors also use genetic inference-time scaling to enhance performance. The model is trained on a dataset of 208.62 million unique images, and the theoretical training cost is approximately 400,000 dollars.

The results show that Boogu-Image-0.1 consistently matches or surpasses other open-source models across standard benchmarks and achieves results approaching leading closed-source systems. The authors share practical discussions and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified multimodal understanding and generation. The code is available on GitHub.

Overall, the paper contributes to the development of open-source multimodal models that can achieve competitive performance with closed-source systems, and provides a valuable resource for the research community.


📅 Published on Jul 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.13125
• PDF: https://arxiv.org/pdf/2607.13125
• Project Page: https://boogu.org/

🤖 Models citing this paper:
• https://huggingface.co/Boogu/Boogu-Image-0.1-Edit
• https://huggingface.co/Boogu/Boogu-Image-0.1-Base
• https://huggingface.co/Boogu/Boogu-Image-0.1-Turbo

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/RioShiina/ImageGen
• https://huggingface.co/spaces/multimodalart/Boogu-Image
• https://huggingface.co/spaces/Tinchote/ImageGen

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #TextToImageGeneration #OpenSourceAI #MultimodalGeneration #InstructionBasedEditing
AI & ML Papers
Photo
🔥 TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training

💡 The paper presents TorchUMM, a unified codebase for evaluating and analyzing multimodal models that can understand, generate, and edit across visual and textual modalities. The development of unified multimodal models has been hindered by the diversity of model architectures and training paradigms, making it challenging to compare and analyze different models. To address this issue, TorchUMM provides a comprehensive framework that supports a wide range of models and tasks, including multimodal understanding, generation, and editing. The codebase includes a broad spectrum of models and integrates both established and novel datasets to evaluate perception, reasoning, compositionality, and instruction-following abilities. By providing a unified interface and standardized evaluation protocols, TorchUMM enables fair and reproducible comparisons across heterogeneous models, facilitating the development of more capable unified multimodal systems. The codebase is designed to foster deeper insights into the strengths and limitations of different models, and its availability is expected to accelerate the development of unified multimodal models. The TorchUMM codebase is available for use, allowing researchers to evaluate and analyze their models in a standardized and reproducible manner. Overall, the paper contributes to the development of unified multimodal models by providing a unified framework for evaluation, analysis, and post-training, which can help to advance the field of multimodal modeling.


📅 Published on Apr 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2604.10784
• PDF: https://arxiv.org/pdf/2604.10784
• Project Page: https://aifrontierlab.github.io/TorchUMM/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalModeling #UnifiedCodebase #MultimodalUnderstanding #VisualTextualModalities #MultimodalGeneration
❤1