Co-Evolving LLM Agents Master Long-Horizon Tasks with Skill Banks

The introduction of the COSPLAY framework marks a substantial advancement in enabling large language models (LLMs) to effectively manage and execute long-horizon tasks within interactive environments. A core limitation of current LLM agents is their struggle with consistent multi-step reasoning and skill chaining over extended periods. This new co-evolutionary approach directly addresses this by allowing an LLM decision agent to dynamically retrieve skills from a learnable skill bank, while a separate skill bank agent continuously discovers, refines, and updates these reusable skills from the agent's experiences. This dual-agent architecture provides a robust mechanism for skill acquisition, retention, and adaptive application.

The framework's efficacy is demonstrated through experiments across six game environments, where an 8B base model utilizing COSPLAY achieved over a 25.1% average reward improvement compared to four frontier LLM baselines. This quantitative leap highlights the critical role of structured skill management in enhancing LLM performance on complex, sequential tasks that demand robust decision-making under partial observability and delayed rewards. The concept of "skill contracts" further refines this process, ensuring that learned skills are well-defined and appropriately applied, thereby improving both the decision-making process and the quality of action generation. This moves beyond simple prompt engineering to a more dynamic, self-improving agent architecture.

The implications of this research extend beyond game environments, pointing towards a future where AI agents can operate more autonomously and effectively in real-world scenarios requiring sustained reasoning and adaptive behavior. By providing LLMs with a mechanism to build and evolve their own skill sets, this work paves the way for more capable robotic systems, intelligent assistants, and complex simulation agents. The ability to discover and reuse skills across episodes suggests a path towards more generalizable and efficient AI learning. However, the dynamic nature of skill evolution also necessitates robust mechanisms for oversight and verification to ensure that learned behaviors remain aligned with intended objectives and safety protocols.