Cloudflare optimized Pingora's memory by 100 TB

Company Cloudflare implemented mathematical optimization in load distribution algorithms of its backend router Pingora. The changes allowed reducing RAM consumption by 100 TB across the company's entire global network.

Engineers reworked the consistent hashing mechanism used to distribute requests between servers. Previously, the system stored redundant hash ranges for each node, leading to unjustifiably high RAM consumption. The new approach uses mathematical approximation to calculate ranges "on the fly", eliminating the need to store huge correspondence tables in RAM.

As a result of the optimization, it was possible to reduce the load on servers without losing traffic balancing accuracy. Using mathematical methods instead of storing static data structures made it possible to release significant hardware resources, which are now redistributed to perform other tasks within the Pingora infrastructure.

This update is part of the strategy to improve the efficiency of Cloudflare's distributed systems. Reducing memory consumption by 100 TB allows the company to scale network bandwidth without the need to purchase additional server equipment, which is critical for handling growing volumes of HTTP traffic.