🔥 ASI-Bench: At the Dawn of Artificial Superintelligence

💡 The paper addresses the gap between current artificial intelligence systems, which are largely built to learn, compress and apply existing human knowledge, and the capabilities required for artificial superintelligence, which must explore unknown domains, generate new knowledge and produce verifiable results with little or no human direction. Existing benchmarks only test whether AI can answer questions based on learned knowledge or follow extensive human instructions, leaving the ability to conduct independent scientific research largely unmeasured.

To fill this gap the authors introduce ASI‑Bench, the first benchmark designed to evaluate AI systems on innovative exploration and autonomous scientific execution across a wide range of research areas. ASI‑Bench comprises sixty project‑level tasks drawn from eleven scientific domains. Each task is built to require the AI to select appropriate methods, conduct the research, and generate results that can be independently verified. The benchmark reduces the amount of methodological guidance provided to the AI in three stages: full logical guidance, only the method name specified, and no guidance at all, forcing the system to determine the method itself. All tasks undergo expert review, AI‑assisted auditing, sandboxed execution, and scorer validation to ensure fairness and reproducibility.

The results show a clear decline in performance as guidance is removed. With full logical guidance the average score across 18 state‑of‑the‑art model configurations is 50.91. When only the method is specified the average drops to 29.10, and when agents must choose the method themselves the average falls further to 26.62. This sharp decline demonstrates that current AI systems remain heavily dependent on human direction and are far from being able to conduct end‑to‑end scientific projects autonomously. The authors make ASI‑Bench publicly available and invite researchers and developers to contribute new tasks, push the limits of today’s AI, and accelerate the collective progress toward artificial superintelligence.

📅 Published on Aug 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2608.17271
• PDF: https://arxiv.org/pdf/2608.17271
• Project Page: https://asibench.apexin.ai/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/Apexintelligence-AI/ASI-Bench-seed31415
• https://huggingface.co/datasets/Apexintelligence-AI/ASI-Bench-seed42

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ArtificialSuperintelligence #ASIBench #AIResearchBenchmark #AutonomousScience #ExploratoryAI