Developers introduced the Laya-coreml engine for Apple Silicon

Developers have released Laya-coreml — a specialized engine for local execution of models with open weights on Apple Silicon devices. The tool uses the capabilities of the Core ML framework and the Apple Neural Engine built into the chips to optimize calculations. In tests on the M3 Max chip, the solution demonstrated a delay of 4.98 ms (P50 indicator). The developers claim that the tool's energy efficiency is 2.78 times higher compared to similar tasks performed through the MLX library. The project is focused on performing typed decision-making tasks directly on the user's hardware. The use of Neural Engine hardware acceleration allows reducing the load on the graphics processor and central CPU, which is critical for autonomous operation of applications on macOS. The project code is available in an open repository on GitHub under a license that allows integration into third-party local AI services. The tool is designed for developers who require high inference speed with minimal battery consumption on Apple laptops. The release expands the ecosystem of tools for local AI, offering an alternative to existing MLX-based solutions. The transition to native Core ML allows for more efficient distribution of computational resources when working with models that require low latency.