✨ Title: When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought

📝 Summary:
MIRA is a new benchmark for evaluating models that use intermediate visual images to enhance reasoning. It includes 546 multimodal problems requiring models to generate and utilize visual cues. Experiments show models achieve a 33.7% performance gain with visual cues compared to text-only prompts...

🔹 Publication Date: Published on Nov 4

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.02779
• PDF: https://arxiv.org/pdf/2511.02779

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VisualReasoning #ChainOfThought #MultimodalAI #AIBenchmark #ComputerVision

✨TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models

📝 Summary:
TiViBench is a new benchmark assessing image-to-video models reasoning across four dimensions and 24 tasks. Commercial models show stronger reasoning potential. VideoTPO, a test-time strategy, significantly enhances performance, advancing reasoning in video generation.

🔹 Publication Date: Published on Nov 17

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.13704
• PDF: https://arxiv.org/pdf/2511.13704
• Project Page: https://haroldchen19.github.io/TiViBench-Page/
• Github: https://haroldchen19.github.io/TiViBench-Page/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoGeneration #AIBenchmark #ComputerVision #DeepLearning #AIResearch

✨Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image

📝 Summary:
MMRB2 is a new benchmark for multimodal reward models, evaluating them on interleaved image and text tasks using 4,000 expert-annotated preferences. It shows top models like Gemini 3 Pro achieve 75-80% accuracy, still below human performance, highlighting areas for improvement in these models.

🔹 Publication Date: Published on Dec 18

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.16899
• PDF: https://arxiv.org/pdf/2512.16899
• Github: https://github.com/facebookresearch/MMRB2/tree/main

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MultimodalAI #RewardModels #AIbenchmark #MachineLearning #AIResearch

✨A Benchmark and Agentic Framework for Omni-Modal Reasoning and Tool Use in Long Videos

📝 Summary:
This paper introduces LongShOTBench, a diagnostic benchmark for long-form multimodal video understanding with open-ended questions and agentic tool use. It also presents LongShOTAgent, an agentic system for video analysis. Results show state-of-the-art models struggle significantly, highlighting ...

🔹 Publication Date: Published on Dec 18

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.16978
• PDF: https://arxiv.org/pdf/2512.16978
• Project Page: https://mbzuai-oryx.github.io/LongShOT/
• Github: https://github.com/mbzuai-oryx/longshot

✨ Datasets citing this paper:
• https://huggingface.co/datasets/MBZUAI/longshot-bench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoAI #MultimodalAI #AgenticAI #AIbenchmark #AIResearch

✨CL-bench: A Benchmark for Context Learning

📝 Summary:
Current LMs struggle with context learning, requiring new knowledge and reasoning beyond pre-training. The CL-bench, a new real-world benchmark, reveals models solve only 17.2 percent of tasks, showing a critical bottleneck for complex real-world applications.

🔹 Publication Date: Published on Feb 3

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.03587
• PDF: https://arxiv.org/pdf/2602.03587
• Project Page: https://www.clbench.com
• Github: https://github.com/Tencent-Hunyuan/CL-bench

✨ Datasets citing this paper:
• https://huggingface.co/datasets/tencent/CL-bench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#ContextLearning #LanguageModels #AIBenchmark #NLP #AIResearch

✨AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents

📝 Summary:
AIRS-Bench is a new benchmark of 20 scientific tasks evaluating AI agents across the full research lifecycle. Agents exceed human state-of-the-art in 4 tasks but largely fall short, highlighting significant room for improvement in autonomous scientific research. The suite is open-sourced to accel...

🔹 Publication Date: Published on Feb 6

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.06855
• PDF: https://arxiv.org/pdf/2602.06855
• Github: https://github.com/facebookresearch/airs-bench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#AIagents #ScientificResearch #AIBenchmark #FrontierAI #AutonomousResearch

✨CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges

📝 Summary:
Researchers introduced CreativeBench, a benchmark for evaluating machine creativity in code generation using a quality-novelty metric. They found scaling improves combinatorial creativity but yields diminishing returns for exploration. They also proposed EvoRePE, an inference-time strategy to enh...

🔹 Publication Date: Published on Mar 12

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.11863
• PDF: https://arxiv.org/pdf/2603.11863
• Project Page: https://zethwang.github.io/creativebench.github.io/
• Github: https://github.com/ZethWang/CreativeBench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MachineCreativity #CodeGeneration #AIBenchmark #GenerativeAI #AIResearch

✨VAREX: A Benchmark for Multi-Modal Structured Extraction from Documents

📝 Summary:
VAREX is a multimodal benchmark for structured data extraction from government forms. It provides four input modalities per document to systematically assess how input format affects extraction accuracy. Key findings show layout-preserving text significantly boosts accuracy and output compliance ...

🔹 Publication Date: Published on Mar 16

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.15118
• PDF: https://arxiv.org/pdf/2603.15118
• Project Page: https://udibarzi.github.io/varex-bench/
• Github: https://github.com/udibarzi/varex-bench

✨ Datasets citing this paper:
• https://huggingface.co/datasets/ibm-research/VAREX

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#DataExtraction #MultimodalAI #DocumentAI #AIbenchmark #NLP

✨MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios

📝 Summary:
A new multilingual document parsing benchmark reveals significant performance gaps between closed-source and open-source models, especially on non-Latin scripts and photographed documents. AI-generate...

🔹 Publication Date: Published on Mar 30

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.28130
• PDF: https://arxiv.org/pdf/2603.28130
• Github: https://github.com/Yuliang-Liu/MultimodalOCR

✨ Datasets citing this paper:
• https://huggingface.co/datasets/Delores-Lin/MDPBench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MultilingualNLP #DocumentAI #OCR #AIbenchmark #MachineLearning

✨AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents

📝 Summary:
Computer-use agents pose unique safety risks as harm can emerge from sequences of individually benign actions. AgentHazard is a benchmark with 2,653 instances to evaluate this. Experiments reveal current systems are highly vulnerable, showing model alignment alone doesnt ensure agent safety.

🔹 Publication Date: Published on Apr 3

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.02947
• PDF: https://arxiv.org/pdf/2604.02947
• Project Page: https://yunhao-feng.github.io/AgentHazard/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#AISafety #AgentAI #AIVulnerability #AIethics #AIbenchmark