Anthropic has introduced Message Threads to speed up work with long context

Anthropic has introduced the Message Threads feature for its Messages API. The new solution optimizes the data transfer process when working with chatbots and AI agents, eliminating the need to resend the entire conversation history with every new request.

Instead of transmitting the full dialogue context at each stage, the system now streams only the fragments that have been changed or added. This change significantly reduces network load and speeds up data processing. According to the developer, for requests ranging from 500,000 to 1,000,000 tokens, the time to first token (TTFT) is reduced by approximately 35%.

The technology is aimed at developers building complex applications with long conversation sessions, where accumulated context previously slowed down model response. Using Message Threads allows for a significant increase in the responsiveness of interfaces working with large volumes of data, without the need to truncate dialogue history or sacrifice the quality of the model's responses.