Co-Evolving LLM Agents Master Long-Horizon Tasks with Skill Banks

The development of COSPLAY, a co-evolutionary framework for LLM decision agents and skill banks, represents a significant advancement in enabling AI to tackle long-horizon tasks requiring multi-step reasoning and robust decision-making. Traditional LLM agents often falter in such complex environments due to their inability to consistently discover, retain, and reuse structured skills across episodes. This new architecture directly addresses that limitation by creating a symbiotic relationship where a decision agent retrieves relevant skills from a dynamic skill bank, while a separate skill bank agent continuously extracts, refines, and updates these reusable skills from the agent's unlabeled interactions.

The technical innovation lies in this dual-agent approach, which allows for simultaneous improvement of both components. The decision agent learns more effective skill retrieval and action generation, while the skill bank agent autonomously curates a valuable repository of reusable behaviors. Experimental validation across six diverse game environments demonstrated the framework's efficacy. Notably, COSPLAY, utilizing an 8B base model, achieved an average reward improvement exceeding 25.1% compared to four leading LLM baselines in single-player game benchmarks. Furthermore, it maintained competitive performance in more complex multi-player social reasoning games, highlighting its versatility and robustness in varied interactive settings.

The implications for autonomous agent development are substantial. This framework provides a scalable method for agents to acquire and leverage complex behavioral repertoires, moving beyond brittle, single-task solutions. It paves the way for more general-purpose AI agents capable of operating effectively in dynamic, partially observable environments. Future research will likely focus on extending this co-evolutionary paradigm to real-world robotic control and complex simulation tasks, where the ability to autonomously learn and adapt skills over extended periods is paramount. This approach could accelerate the deployment of AI in domains requiring sophisticated, adaptive long-term planning and execution.