Developer runs Kimi K3 model on a standard PC using C99

An enthusiast has introduced Kimi K3 in C — an implementation of a 2.78 trillion parameter language model in C99. The project allows for running inference on a model with a total checkpoint size of 1.56 TB on a standard personal computer equipped with only 8 GB of RAM.

A key feature of the solution is a mechanism for streaming data directly from storage into memory. Instead of loading the entire model at once, the system reads the necessary weights as calculations are performed, which bypasses hardware limitations. The code is written in pure C, minimizing execution overhead.

The implementation is designed to work with the Kimi K3 model architecture. Using disk space as an extension of RAM allows for running extremely heavy models on consumer configurations where VRAM or RAM capacity would physically not allow for loading the weights in full.

For practitioners, this means the ability to run models locally that previously required clusters with tens of terabytes of memory. Despite the expectedly low generation speed due to disk bandwidth limitations, the project demonstrates the viability of the "disk inference" method for working with giant neural networks on accessible hardware.