Researchers presented the RBS-Attention method to accelerate the prefill of LLM

A group of researchers developed RBS-Attention — a method of sparse prefill that solves the problem of "average blur" when processing long contexts in large language models. The technology uses a two-branch block selection system: the base branch determines the average relevance, and the "rescue" branch identifies tokens that may be mistakenly ignored by standard sparse attention methods. The method does not require additional training and maintains compatibility with the FlashAttention mechanism. During testing on the H100 GPU when working with a context of 128K on the Qwen3-30B-A3B-Instruct-2507-FP8 model, the solution demonstrated 20.65-fold acceleration of standalone prefill and 11.92-fold acceleration when using the vLLM engine. The time-to-first-token (TTFT) metric was reduced by 5.97 times. The model's accuracy when using RBS-Attention remains comparable to dense attention. On the RULER benchmark, the Qwen3-32B model showed a result of 88.65 versus 89.52 for the standard architecture. Additional quality checks were carried out on the LongBench-v2, InfiniteBench, and Video-MME datasets. The use of sparse attention allows significantly reducing computational costs at the prompt processing stage, which is a "bottleneck" when working with long documents. The new approach allows dynamically adapting block selection depending on the layer, attention head, and structure of the prompt itself, making it an effective tool for deploying models with long context support in production.