The CADWorld benchmark is presented for evaluating AI agents in FreeCAD

Researchers introduced CADWorld — a specialized benchmark for testing AI agents operating in professional engineering environments. The system evaluates the ability of models to perform complex tasks in FreeCAD, requiring prolonged interaction with the graphical interface and adherence to strict geometric constraints. The benchmark includes 200 tasks, divided into 11 categories, including part modeling, assembly, CAM processing, FEM analysis, and creation of technical drawings. Unlike standard tests, success in CADWorld is determined not by visual similarity, but by automated checks of parameters, project structure, and correctness of production data. Testing of seven current models showed a significant gap between the current capabilities of AI and industry requirements. The best agent achieved only 17.5% success, while the expert level is 87.0%. Weak models often fail to create valid files, and more advanced systems make errors in structural and geometric requirements. Developers note that CADWorld clearly demonstrates the lack of skills in modern agents in tasks that require long-term planning and preservation of the integrity of engineering artifacts. The toolkit is available for use under the Creative Commons Attribution 4.0 license, allowing researchers to integrate it into their testing pipelines.