New InteractWeb-Bench Reveals MLLM Agents Struggle with Real-World Website Generation

The introduction of InteractWeb-Bench represents a significant stride in the rigorous evaluation of multimodal large language models (MLLMs) for complex, real-world applications such as website generation. This benchmark critically exposes a pervasive failure mode termed 'blind execution,' where advanced AI agents struggle to interpret and adapt to ambiguous, low-quality instructions from non-expert users. This directly challenges the idealized assumptions often present in existing benchmarks, which typically rely on well-structured inputs and static execution environments, thereby failing to capture the true complexities of human-AI interaction.

InteractWeb-Bench is meticulously designed to simulate diverse user behaviors through four distinct user agents and persona-driven instruction perturbations, encompassing ambiguity, redundancy, and contradiction. Its innovative interactive execution environment, featuring a unified action space (Clarify, Implement, Verify, Submit), allows for iterative intent refinement and visual feedback-based validation. The empirical findings are stark: even frontier MLLM-based agents remain ensnared in blind execution, highlighting profound limitations in their intent recognition capabilities and their capacity for adaptive interaction. This research underscores a critical gap between theoretical AI advancements and practical deployability in user-centric creative domains.

The implications for the future trajectory of AI agent development are profound. Overcoming 'blind execution' necessitates a fundamental shift in MLLM architecture and training methodologies, prioritizing robust intent understanding, proactive clarification, and dynamic adaptation to evolving user requirements. This benchmark provides a clear, actionable framework for researchers to focus on developing agents that can engage in more sophisticated, human-like dialogue and iterative refinement, ultimately paving the way for more reliable and user-friendly AI tools in web development and beyond. The open-sourced nature of InteractWeb-Bench further ensures community-driven progress in this crucial area.