💡 The paper introduces RF-DETR, a lightweight detection transformer that uses neural architecture search to optimize accuracy and latency for real-time object detection. The motivation behind this work is that current state-of-the-art detectors often fail to generalize to real-world datasets with classes not seen during pre-training. Instead of fine-tuning a large vision-language model, the authors propose RF-DETR, which fine-tunes a pre-trained base network on a target dataset and evaluates thousands of network configurations to find the best accuracy-latency tradeoff. The approach uses weight-sharing neural architecture search to improve transferability to diverse target domains. The results show that RF-DETR significantly outperforms prior state-of-the-art real-time methods on several datasets, including COCO and Roboflow100-VL. Specifically, RF-DETR achieves 48.0 AP on COCO, beating a similar method by 5.3 AP at similar latency, and RF-DETR also outperforms another method on Roboflow100-VL while running 20 times as fast. Notably, RF-DETR is the first real-time detector to surpass 60 AP on COCO, demonstrating its effectiveness in achieving high accuracy and low latency. The code for RF-DETR is made available, allowing for further research and development in the field of real-time object detection.