Anthropic cuts Claude Code latency with server-side message threads

Anthropic has launched message threads, a new feature for its Messages API designed to accelerate long-running AI sessions by reducing redundant data transmission. Available now in public beta, the update is already active for most Claude Code users and aims to cut the time it takes for the model to start responding.

What happened

The core issue addressed by this update is the inefficiency of traditional API interactions during extended tasks. Previously, every request sent to Claude required the client to upload the entire conversation history, including all prior messages, tool calls, and system prompts. As a coding agent works through complex tasks, reading files and making edits, this history grows rapidly. By the time a session reaches 500,000 tokens, each new request involves re-uploading that massive amount of data, which the API must then validate and tokenize before processing can begin.

Message threads change this dynamic by allowing Anthropic to temporarily retain conversation history on its servers. The initial request still sends the full context, but subsequent requests only need to include new messages and a reference to the previous response ID. Anthropic reconstructs the full conversation server-side before passing it to the model. This shift has reduced the median time to first token by approximately 35% for Claude Code requests containing between 500,000 and 1 million tokens.

While the feature significantly lowers latency and data transfer volume, it does not reduce API costs. Anthropic continues to charge for all tokens processed by the model, regardless of how much data the client uploads. The company notes that developers can combine message threads with prompt caching to potentially lower costs, as the two features address different aspects of performance and billing.

How it works

Technically, message threads introduce a new parameter to the Messages API. To start a thread, developers include thread={"type": "create"} in their initial request along with the full conversation history. For follow-up turns, they use thread={"type": "continue", "previous_message_id": [ID]}, sending only the new input. The API retains the earlier context, system prompts, and tool definitions in server memory, eliminating the need for repeated uploads.

For those using the Python SDK, a thread helper simplifies this process. By initializing a thread via client.beta.messages.thread(), the SDK automatically manages the creation and continuation logic, including adding the required beta header anthropic-beta: message-threads-2026-08-12. This abstraction allows developers to focus on sending messages via thread.send() without manually tracking response IDs or handling low-level API parameters.

Key details

• Message threads are available in public beta for all Claude models, including those on AWS, Microsoft Foundry, and Google Cloud.
• The feature reduces median time to first token by roughly 35% for large contexts between 500,000 and 1 million tokens.
• Threads expire after five minutes of inactivity, requiring a full history resend if the pause exceeds this limit.
• Token usage and billing remain unchanged because Claude still processes the entire conversation history on every turn.
• Thread contents are encrypted and stored in server memory for up to an hour after last use, but are not written to persistent storage.
• The feature is currently incompatible with zero data retention policies and HIPAA compliance requirements.

Why it matters

For engineers building AI agents, particularly those involving computer use or long-running coding tasks, latency is a critical bottleneck. Guillaume Lebedel, CTO of StackOne, illustrated the potential impact by estimating that a computer-use agent capturing screenshots could reduce cumulative data uploads from 191 MB to 7.5 MB over a 50-step session. This drastic reduction in network payload means faster round-trip times and more responsive agents, even if the computational cost on Anthropic’s side remains the same.

However, the five-minute expiry window introduces a constraint that developers must account for in their application logic. While continuous agents will rarely hit this limit, interactive tools used by humans may face interruptions. If a developer steps away for a coffee break, the thread expires, and the next request must re-upload the full history. Although the SDK handles this gracefully by establishing a new thread, the initial delay after a break will be higher until the thread is re-established.

What you can do

• Update your Anthropic Python SDK to the latest version to access the client.beta.messages.thread() helper.
• Refactor long-running agent loops to use the thread helper, ensuring you pass the betas=["message-threads-2026-08-12"] header if making direct API calls.
• Monitor your agent’s idle times and implement local state management to handle thread expiry gracefully when resuming sessions.
• Combine message threads with prompt caching for static content like system instructions to optimize both latency and cost.
• Review your data compliance requirements, as message threads are not currently suitable for HIPAA-regulated or zero-retention environments.
• Test the feature with large-context workloads to measure the actual improvement in time to first token for your specific use case.