✨Mitigating Object and Action Hallucinations in Multimodal LLMs via Self-Augmented Contrastive Alignment

📝 Summary:
The SANTA framework addresses object and action hallucinations in multimodal LLM video captions. It uses self-augmented contrastive alignment to identify potential hallucinations and then aligns regional objects and actions with visual phrases, improving factual accuracy. Experiments show SANTA o...

🔹 Publication Date: Published on Dec 4

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.04356
• PDF: https://arxiv.org/pdf/2512.04356
• Project Page: https://kpc0810.github.io/santa/
• Github: https://kpc0810.github.io/santa/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MultimodalLLMs #AI #Hallucinations #VideoUnderstanding #ContrastiveLearning

✨Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding

📝 Summary:
Active Video Perception AVP improves long video understanding by actively seeking query-relevant evidence. It uses an iterative plan-observe-reflect process, acquiring compact evidence directly from pixels. This achieves higher accuracy with reduced computational cost.

🔹 Publication Date: Published on Dec 5

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.05774
• PDF: https://arxiv.org/pdf/2512.05774

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoUnderstanding #ActiveLearning #ComputerVision #AIResearch #DeepLearning

✨LongVideoAgent: Multi-Agent Reasoning with Long Videos

📝 Summary:
A multi-agent framework with a master LLM, grounding agent, and vision agent enhances long-video QA by improving temporal grounding and extracting visual details. This RL-trained system outperforms non-agent baselines on new datasets.

🔹 Publication Date: Published on Dec 23

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.20618
• PDF: https://arxiv.org/pdf/2512.20618
• Github: https://longvideoagent.github.io/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MultiAgentSystems #LLM #VideoUnderstanding #ComputerVision #AI

✨Taming Hallucinations: Boosting MLLMs' Video Understanding via Counterfactual Video Generation

📝 Summary:
MLLMs struggle with hallucinations on counterfactual videos. DualityForge synthesizes counterfactual video data and QA pairs through diffusion-based editing to address this. This method significantly reduces model hallucinations and improves general performance.

🔹 Publication Date: Published on Dec 30, 2025

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.24271
• PDF: https://arxiv.org/pdf/2512.24271
• Project Page: https://amap-ml.github.io/Taming-Hallucinations/
• Github: https://github.com/AMAP-ML/Taming-Hallucinations

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MLLMs #VideoUnderstanding #AIHallucinations #GenerativeAI #MachineLearning

✨LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization

📝 Summary:
LongVPO is a two-stage DPO framework for short-context VLMs to understand long videos. It uses synthetic preference data from anchored clips and recursive captioning for multi-segment reasoning. LongVPO achieves state-of-the-art with minimal human annotation.

🔹 Publication Date: Published on Feb 2

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.02341
• PDF: https://arxiv.org/pdf/2602.02341
• Github: https://github.com/MCG-NJU/LongVPO

🔹 Models citing this paper:
• https://huggingface.co/MCG-NJU/LongVPO-Stage2-InternVL3-8B
• https://huggingface.co/MCG-NJU/LongVPO-Stage1-InternVL3-8B

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoUnderstanding #MachineLearning #VLMs #DeepLearning #AIResearch

✨SemanticMoments: Training-Free Motion Similarity via Third Moment Features

📝 Summary:
Existing video models struggle with semantic motion often biased towards appearance. SemanticMoments addresses this with a training-free method using temporal statistics on semantic features to consistently outperform other approaches for motion-centric video understanding.

🔹 Publication Date: Published on Feb 9

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.09146
• PDF: https://arxiv.org/pdf/2602.09146
• Project Page: https://x.com/HubermanSaar/status/2023485404280672498?s=20

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SemanticMoments #VideoUnderstanding #ComputerVision #MachineLearning #MotionAnalysis

✨LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding

📝 Summary:
LongVideo-R1 is an MLLM agent for efficient long video understanding with low computational cost. It uses active reasoning and selective clip navigation, avoiding exhaustive search by focusing on informative segments. This achieves superior accuracy and efficiency.

🔹 Publication Date: Published on Feb 24

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.20913
• PDF: https://arxiv.org/pdf/2602.20913
• Github: https://github.com/qiujihao19/LongVideo-R1

🔹 Models citing this paper:
• https://huggingface.co/ChurchillQAQ/LongVideo-R1-Qwen2.5
• https://huggingface.co/ChurchillQAQ/LongVideo-R1-Qwen3

✨ Datasets citing this paper:
• https://huggingface.co/datasets/ChurchillQAQ/LongVideo-R1-Data

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoUnderstanding #MLLM #AI #EfficientAI #DeepLearning

✨Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

📝 Summary:
Video Streaming Thinking VST is a novel paradigm for real-time video understanding, enabling AI to think while watching during streaming playback. It optimizes VideoLLMs for responsive, low-latency interaction, showing significantly faster responses and strong performance on various benchmarks.

🔹 Publication Date: Published on Mar 12

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.12262
• PDF: https://arxiv.org/pdf/2603.12262
• Project Page: https://1ranguan.github.io/VST/
• Github: https://github.com/1ranGuan/VST

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoLLMs #RealTimeAI #VideoUnderstanding #AIResearch #MachineLearning

✨V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning

📝 Summary:
V-JEPA 2.1 is a self-supervised model learning dense visual representations for images and videos. It combines dense predictive loss, deep self-supervision, multi-modal tokenizers, and scaling to achieve state-of-the-art performance across various benchmarks, significantly advancing visual unders...

🔹 Publication Date: Published on Mar 15

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.14482
• PDF: https://arxiv.org/pdf/2603.14482
• Project Page: https://ai.meta.com/blog/v-jepa-2-world-model-benchmarks/
• Github: https://github.com/facebookresearch/vjepa2

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SelfSupervisedLearning #ComputerVision #DeepLearning #AI #VideoUnderstanding

✨CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Management

📝 Summary:
CurveStream enhances streaming video understanding in MLLMs via a curvature-aware hierarchical memory framework. It dynamically routes frames based on semantic intensity to prevent Out-of-Memory errors and achieve over 10 percent performance gains.

🔹 Publication Date: Published on Mar 20

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.19571
• PDF: https://arxiv.org/pdf/2603.19571
• Github: https://github.com/streamingvideos/CurveStream

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MLLMs #StreamingVideo #VideoUnderstanding #MemoryManagement #AI

✨StreamingClaw Technical Report

📝 Summary:
StreamingClaw is a unified framework for real-time streaming video understanding and embodied intelligence. It integrates real-time reasoning, multimodal long-term memory, and proactive interaction, enabling direct control of the physical world.

🔹 Publication Date: Published on Mar 23

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.22120
• PDF: https://arxiv.org/pdf/2603.22120

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#EmbodiedAI #VideoUnderstanding #RealTimeAI #Robotics #MultimodalAI

✨AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding

📝 Summary:
AdaptToken enables efficient long video understanding for MLLMs by using model uncertainty to dynamically select relevant tokens. It allocates a global token budget and supports early stopping, significantly improving accuracy and reducing inference time across benchmarks.

🔹 Publication Date: Published on Mar 30

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.28696
• PDF: https://arxiv.org/pdf/2603.28696
• Project Page: https://haozheqi.github.io/adapt-token
• Github: https://github.com/HaozheQi/AdaptToken

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MLLM #VideoUnderstanding #MachineLearning #AIResearch #TokenSelection

✨A Simple Baseline for Streaming Video Understanding

📝 Summary:
A simple sliding-window approach outperforms complex memory-based streaming video methods by using only recent frames. It demonstrates a trade-off between real-time perception and long-term memory, suggesting benchmarks should separate these abilities.

🔹 Publication Date: Published on Apr 2

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.16655
• PDF: https://arxiv.org/pdf/2604.02317
• Project Page: https://simple-stream.github.io/
• Github: https://simple-stream.github.io/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoUnderstanding #StreamingAI #ComputerVision #RealTimeAI #MachineLearning

✨AURA: Always-On Understanding and Real-Time Assistance via Video Streams

📝 Summary:
AURA is an end-to-end streaming visual interaction framework for continuous video understanding. It enables real-time question answering and proactive responses, improving on current VideoLLMs through integrated context management and optimized deployment.

🔹 Publication Date: Published on Apr 5

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.04184
• PDF: https://arxiv.org/pdf/2604.04184
• Project Page: https://aurateam2026.github.io
• Github: https://github.com/aurateam2026/AURA

🔹 Models citing this paper:
• https://huggingface.co/aurateam/AURA

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoUnderstanding #RealTimeAI #VideoLLM #ComputerVision #DeepLearning

✨Watch Before You Answer: Learning from Visually Grounded Post-Training

📝 Summary:
VLMs struggle with video understanding due to text biases in benchmarks and training data. VidGround uses only visually grounded questions for post-training to eliminate these biases. This improves VLM performance and emphasizes the need for high-quality, visually grounded data.

🔹 Publication Date: Published on Apr 6

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.05117
• PDF: https://arxiv.org/pdf/2604.05117
• Project Page: http://vidground.etuagi.com
• Github: https://github.com/reacher-z/vidground

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VLMs #VideoUnderstanding #AI #MachineLearning #ComputerVision

✨Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator

📝 Summary:
Uni-ViGU introduces a unified framework for video generation and understanding, uniquely building upon a video generator as its foundation. It uses unified flow matching and a bidirectional training mechanism to achieve competitive performance in both generation and understanding tasks.

🔹 Publication Date: Published on Apr 9

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.08121
• PDF: https://arxiv.org/pdf/2604.08121
• Project Page: https://fr0zencrane.github.io/uni-vigu-page/
• Github: https://fr0zencrane.github.io/uni-vigu-page/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoGeneration #VideoUnderstanding #DiffusionModels #AIResearch #DeepLearning

🔥 HumanNet: Scaling Human-centric Video Learning to One Million Hours

💡 The paper introduces HumanNet, a large-scale human-centric video dataset that captures how humans interact with the physical world, with the goal of advancing embodied intelligence. The problem addressed is the lack of large, diverse, and richly annotated human activity data, which hinders progress in learning physical interaction. To solve this, the authors created a one-million-hour video corpus that spans first-person and third-person perspectives, covering various activities, human-object interactions, and long-horizon behaviors in diverse environments. The dataset is annotated with interaction-centric information, including captions, motion descriptions, and hand and body-related signals.

The method involves a systematic data curation paradigm that treats human-centric filtering, temporal structuring, viewpoint diversity, and annotation enrichment as key design principles. This approach transforms unstructured internet video into a scalable substrate for representation learning, activity understanding, motion generation, and human-to-robot transfer.

The results show that HumanNet can be used to train vision-language-action models, and that egocentric human video can effectively replace robot data for training. In a controlled experiment, the authors found that continued training with 1000 hours of egocentric video from HumanNet surpassed the performance of continued training with 100 hours of real-robot data. This suggests that human-centric video can be a scalable and cost-effective substitute for robot data, and that HumanNet can be used to explore the opportunity to scale embodied foundation models using human-centric videos. Overall, the paper contributes a large-scale dataset and a systematic data curation paradigm that can advance embodied intelligence and learning physical interaction.

📅 Published on May 7

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.06747
• PDF: https://arxiv.org/pdf/2605.06747
• Project Page: https://dagroup-pku.github.io/HumanNet/
• GitHub: https://github.com/DAGroup-PKU/HumanNet ⭐ 65

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#HumanCentricVideoLearning #EmbodiedIntelligence #LargeScaleVideoDatasets #HumanActivityRecognition #VideoUnderstanding

🔥 M^3Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

💡 The paper introduces M3Eval, a comprehensive evaluation framework and benchmark for assessing the memory capabilities of multi-modal models in video understanding systems. The problem addressed is that current multi-modal models have significant limitations in their memory capabilities, particularly in maintaining disentangled representations and demonstrating human-like interference patterns. To address this gap, the authors designed M3Eval, which is grounded in cognitive psychology and features carefully constructed tasks that isolate key aspects of memory.

The method involves conducting extensive experiments across representative multi-modal models using the M3Eval framework, which evaluates different memory dimensions such as what models retain, how faithfully information is preserved, and how robust memory remains under interference. The framework includes tasks that test the models' ability to maintain disentangled representations, exhibit human-like interference patterns, and demonstrate symbolic memory.

The results of the experiments reveal consistent weaknesses and distinctive behaviors in the models. The models struggle to maintain disentangled representations when processing parallel video streams, exhibit interference patterns that differ substantially from those observed in human memory, and ground memory sources more reliably in the spatial domain than the temporal domain. Additionally, the models demonstrate limited symbolic memory.

The paper's contributions include providing a valuable resource for future research in the form of the M3Eval benchmark and highlighting memory as a fundamental yet underexplored capability in multi-modal models. The findings offer insights for designing more effective memory mechanisms in multi-modal models, which can advance the field of video understanding systems. The code and dataset are made available to facilitate future research.

📅 Published on Jun 3

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.05008
• PDF: https://arxiv.org/pdf/2606.05008
• Project Page: https://pku-value-lab.github.io/m3eval-homepage/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/PKU-VaLuE-Lab/m3eval

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #VideoUnderstanding #CognitiveArchitectures #MemoryEvaluation #MultimodalModels

🔥 OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains

💡 The paper introduces a new dataset and method for improving audio-visual question answering systems. Current systems typically process videos in short clips and generate separate descriptions for audio and visual modalities, which can lead to inconsistent descriptions and a lack of cross-modal reasoning. To address this, the authors propose a two-part approach: entity-anchored video scripting, which transforms videos into structured scripts with summaries, main entity lists, and segment-wise audio-visual descriptions, and clue-guided QA generation, which prompts models to mine cross-segment clues from the script and generate QA pairs based on these clues.

The entity-anchored video scripting mechanism ensures cross-segment referential consistency and reconstructs audio-visual associations, while the clue-guided QA generation mechanism encourages models to generate questions that require long-term temporal connections and deep cross-modal reasoning. The authors use this pipeline to construct a new dataset called OmniVideo-100K, which consists of structured scripts and QA pairs, as well as a human-verified test set called OmniVideo-Test.

The results show that fine-tuning models on OmniVideo-100K yields significant performance gains, with improvements of up to 20.59% on the OmniVideo-Test set. The models also demonstrate strong generalization, with improvements of up to 12.64% on established benchmarks such as Daily-Omni and JointAVBench. Overall, the paper contributes a new dataset and method for improving audio-visual question answering systems, with a focus on cross-modal reasoning and temporal consistency.

📅 Published on Jun 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.14702
• PDF: https://arxiv.org/pdf/2606.14702
• Project Page: https://yzlmhzz.github.io/OmniVideo-100K/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MiG-NJU/OmniVideo-100K
• https://huggingface.co/datasets/MiG-NJU/OmniVideo-Test

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AudioVisualReasoning #MultimodalLearning #VideoUnderstanding #CrossModalReasoning #AudioVisualQuestionAnswering

🔥 VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

💡 The paper introduces VideoChat3, a fully open, efficient, and generalist video-centric multimodal large language model for video understanding. Current open-source models are limited in several ways, struggling to generalize across diverse video types and being computationally demanding, which restricts their efficiency and scalability. Most models are also only partially open, with key components such as training code, strategy, or datasets unavailable, hindering reproducibility and slowing community-driven development.

To address these issues, VideoChat3 advances video understanding through two complementary designs. For efficiency, it introduces the Inflated 3D Vision Transformer and Adaptive Frame Resolution for Streaming Video Perception, enabling efficient spatiotemporal representation and reducing the cost of processing video inputs during training and inference. For effectiveness, it develops a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets, covering general, long-form, and streaming video scenarios, which improves the model's generalization across domains.

By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts, using only 4B parameters and achieving higher efficiency. The paper's contributions include a fully open and efficient video understanding model, a scalable video data synthesis pipeline, and state-of-the-art results on various video understanding benchmarks.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.14935
• PDF: https://arxiv.org/pdf/2607.14935
• Project Page: https://mcg-nju.github.io/VideoChat3

🤖 Models citing this paper:
• https://huggingface.co/MCG-NJU/VideoChat3-4B
• https://huggingface.co/MCG-NJU/I3D-ViT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-LV116k
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-Academic2M
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-OL617k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoUnderstanding #MultimodalLearning #LargeLanguageModels #VideoCentricAI #EfficientComputerVision