💡 The paper introduces S-Agent, a spatial reasoning framework that enhances visual language models to enable continuous 3D world understanding from multi-view imagery. The problem addressed is that existing visual language models and tool-augmented agents are limited to static and stateless inference from isolated visual observations, which is insufficient for real-world spatial intelligence.

The S-Agent method involves formulating spatial reasoning as spatio-temporal evidence accumulation, rather than isolated frame-level prediction. This is achieved by casting the visual language model as a semantic planner that decides what evidence is needed, while a hierarchy of spatial tools and experts grounds objects in 2D, lifts them into 3D geometric evidence, and aggregates this evidence into high-level spatial knowledge. The framework also includes a temporal memory mechanism, comprising scene memory and agent memory, which enables evidence integration across frames and reasoning steps.

The results show that S-Agent consistently improves both open-source and closed-source visual language models in a training-free manner. Additionally, supervised fine-tuning on S-Agent-generated spatial trajectories yields S-Agent-8B, a compact spatial agent that significantly surpasses similar-scale baselines and performs comparably to advanced closed-source models. The comprehensive experiments on multi-view and video spatial reasoning benchmarks demonstrate the effectiveness of the S-Agent framework in enhancing spatial intelligence. Overall, the paper contributes a novel spatial tool-use agentic paradigm for understanding and reasoning over continuous multi-view images and videos, which has the potential to improve real-world spatial intelligence applications.