💡 The paper introduces RF-DETR, a lightweight detection transformer that uses neural architecture search to optimize accuracy and latency for real-time object detection. The motivation behind this work is that current state-of-the-art detectors often fail to generalize to real-world datasets with classes not seen during pre-training. Instead of fine-tuning a large vision-language model, the authors propose RF-DETR, which fine-tunes a pre-trained base network on a target dataset and evaluates thousands of network configurations to find the best accuracy-latency tradeoff. The approach uses weight-sharing neural architecture search to improve transferability to diverse target domains. The results show that RF-DETR significantly outperforms prior state-of-the-art real-time methods on several datasets, including COCO and Roboflow100-VL. Specifically, RF-DETR achieves 48.0 AP on COCO, beating a similar method by 5.3 AP at similar latency, and RF-DETR also outperforms another method on Roboflow100-VL while running 20 times as fast. Notably, RF-DETR is the first real-time detector to surpass 60 AP on COCO, demonstrating its effectiveness in achieving high accuracy and low latency. The code for RF-DETR is made available, allowing for further research and development in the field of real-time object detection.

💡 The paper presents Moebius, a lightweight image inpainting framework that achieves high fidelity results with significantly reduced parameters and inference time. The problem addressed is that current industrial foundation models for image inpainting have high computational costs, making them impractical for deployment. To solve this, the authors propose a novel approach that reconstructs the diffusion backbone using local-global interaction blocks and adaptive distillation strategies.

The method involves introducing the Local-λ Mix Interaction block, which consists of Local-λ and Interactive-λ modules. This block summarizes spatial contexts and global semantic priors into fixed-size linear matrices, preserving complex latent interactions while reducing parameters. The authors also propose an adaptive multi-granularity distillation strategy that operates within the latent space to avoid expensive pixel-space decoding. This strategy dynamically balances multiple gradient-based losses to achieve high-fidelity alignment.

The results show that Moebius achieves high-fidelity image inpainting results that rival or surpass those of the 10B-level industrial generalist FLUX.1-Fill-Dev, while using less than 2% of the parameters and delivering a more than 15 times acceleration in total inference time. The Moebius framework has 0.22 billion parameters, compared to 11.9 billion parameters in the industrial generalist model, and achieves a new efficiency standard for high-fidelity inpainting. Overall, the paper presents a highly efficient and optimized image inpainting framework that can be practically deployed.

💡 The paper introduces VideoChat3, a fully open, efficient, and generalist video-centric multimodal large language model for video understanding. Current open-source models are limited in several ways, struggling to generalize across diverse video types and being computationally demanding, which restricts their efficiency and scalability. Most models are also only partially open, with key components such as training code, strategy, or datasets unavailable, hindering reproducibility and slowing community-driven development.

To address these issues, VideoChat3 advances video understanding through two complementary designs. For efficiency, it introduces the Inflated 3D Vision Transformer and Adaptive Frame Resolution for Streaming Video Perception, enabling efficient spatiotemporal representation and reducing the cost of processing video inputs during training and inference. For effectiveness, it develops a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets, covering general, long-form, and streaming video scenarios, which improves the model's generalization across domains.

By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts, using only 4B parameters and achieving higher efficiency. The paper's contributions include a fully open and efficient video understanding model, a scalable video data synthesis pipeline, and state-of-the-art results on various video understanding benchmarks.