Alibaba introduces Co-Design Bench for AI agents

The company Alibaba has developed a new benchmark, Co-Design Bench, designed to evaluate the programming capabilities of AI agents based on real-world user requests. The test includes tasks that have previously caused genuine difficulties in practical application.

The test results revealed a significant gap between laboratory metrics and real-world performance. The best result was demonstrated by the GPT-6 Astra model, which managed to fully solve only 26.4% of the proposed tasks.

On older standard benchmarks, similar models often show scores above 99%. Such discrepancies highlight that existing language models still require significant refinement and manual human intervention in the vast majority of practical development scenarios.