💡 The paper introduces Decompile-Bench, a large-scale open-source dataset designed to improve the accuracy of large language model-based decompilers. The problem addressed is the lack of a comprehensive benchmark for evaluating decompilation technology, which is necessary for converting low-level binaries into human-readable source code. Previous efforts have relied on limited or synthetic benchmarks that do not accurately represent real-world binary-source mappings.

To address this issue, the authors created Decompile-Bench, which consists of two million binary-source function pairs generated from 100 million collected function pairs, totaling 450GB of binaries compiled from permissively licensed GitHub projects. The dataset is accompanied by a benchmark, Decompile-Bench-Eval, which includes manually crafted binaries from established datasets and compiled GitHub repositories released after 2025 to mitigate data leakage issues.

The results show that fine-tuning large language model-based decompilers with Decompile-Bench leads to a 20% improvement in re-executability rate compared to previous benchmarks. The authors also explored commonly used evaluation metrics to provide a thorough assessment of the studied decompilers. The code and data are publicly available on HuggingFace and GitHub, making it a valuable resource for advancing decompilation technology. Overall, Decompile-Bench provides a significant contribution to the field by offering a large-scale, real-world dataset for evaluating and improving decompilation accuracy.