🔥 RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

💡 The paper presents RESOURCE2SKILL, a framework that extracts executable skills for software agents from human-created multimodal resources such as tutorial videos, repositories, articles, and reference artifacts. Existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving multimodal human resources largely underused. RESOURCE2SKILL addresses this issue by distilling these resources into executable skills, which are organized as a hierarchical multimodal Skill Wiki. Each entry in the wiki combines structured text, code, visual examples, metadata, and provenance, preserving complementary signals from different sources.

The framework allows agents to retrieve and compose relevant skills from the wiki at inference time. When coverage is insufficient, the same construction operator can acquire new skills online. The authors evaluate RESOURCE2SKILL across seven practical authoring domains and demonstrate that it improves the average overall score by 11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells.

The results confirm the value of the multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition. The paper's contributions include a novel framework for extracting executable skills from multimodal resources, a hierarchical multimodal Skill Wiki, and a method for online skill acquisition. Overall, RESOURCE2SKILL provides a new approach to creating skill libraries for software agents, leveraging the wealth of information available in human-created multimodal resources.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.29538
• PDF: https://arxiv.org/pdf/2606.29538
• Project Page: https://aka.ms/Resources2Skill

📊 Datasets citing this paper:
• https://huggingface.co/datasets/microsoft/RESOURCE2SKILL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #ExecutableAgentSkills #HumanCentricAI #SkillExtraction #MultimodalKnowledgeRepresentation