Cloudflare fixes cross-request data leak in GLM-5.3 inference engine

Cloudflare disclosed a security incident involving its Workers AI platform where the GLM-5.3 model inadvertently returned data from other users' requests. The vulnerability, active from August 27 to September 9, 2026, affected 155 responses and was triggered by specific long-context prompts. Cloudflare has since patched the underlying inference engine and restored the model with enhanced safety checks.

What happened

On September 8, 2026, security researcher Gadi Evron reported that the @cf/zai-org/glm-5.3 model on Workers AI was leaking session data. Cloudflare reproduced the issue the following morning and took the model offline within 40 minutes. The investigation revealed that between August 27 and September 9, a small number of customers received responses containing text from other customers' requests made to the same model shortly before theirs. All 155 affected responses were generated from prompts exceeding 681,280 tokens. Cloudflare found no evidence of deliberate exploitation, noting that the request patterns matched ordinary long-context usage rather than targeted attacks.

The root cause was a combination of a configuration error and a bug in SGLang, the open-source inference engine powering the service. Cloudflare had deployed GLM-5.3 with a context window of 1,048,576 tokens, but the Key-Value (KV) cache pool on the prefill GPUs could only hold 681,280 tokens. While the team was aware of this mismatch, they assumed the engine would return an error for prompts that did not fit. Instead, the engine attempted to process the request using a flawed rejection path, leading to the exposure. No other models hosted on Workers AI were affected because their KV pools were larger than their respective context lengths.

How it works

Workers AI serves GLM-5.3 using a disaggregated architecture where prefill GPUs process prompts and generate KV caches, which are then transferred via remote direct memory access (RDMA) to decode GPUs for response generation. The KV cache is stored in a shared pool of fixed-size pages in GPU memory. To maintain high performance, these pages are not zeroed out between requests; the engine relies on strict ownership tracking to ensure that a new request overwrites any previous data in its assigned pages.

When a prompt exceeded the KV pool size, the SGLang scheduler marked the request as rejected but replaced it with a one-token placeholder to allow it to drain through the batch machinery. In a single-node setup, this would result in an error. However, in the disaggregated setup, the admission queues checked if a request was "finished" rather than "rejected." Since the rejected request was not marked as finished, it was admitted to the prefill node. The prefill node generated a cache for the single-token placeholder and transferred it to the decode node. The decode node, expecting a full transfer, retained the remaining reserved pages which still contained stale data from previous users. Consequently, the final response included coherent text generated from the placeholder mixed with leaked data from the stale cache pages.

Key details

• Affected Model: Only @cf/zai-org/glm-5.3 was impacted; all other Workers AI models remained secure.
• Exposure Window: The vulnerability was present from August 27, 2026, until the model was taken offline on September 9, 2026.
• Impact Scope: Exactly 155 responses were affected, all originating from prompts longer than 681,280 tokens.
• Root Cause: A mismatch between the 1,048,576-token context window and the 681,280-token KV cache pool, combined with a bug in SGLang’s handling of rejected requests in disaggregated mode.
• Remediation: Cloudflare increased the KV pool size to approximately 2.04 million tokens and patched SGLang to correctly abort rejected requests before KV transfer.
• Upstream Fixes: The SGLang maintainers merged several patches, including PR #38704 and #38935, and released versions v0.5.21 and v0.5.22 to address the issue for all users.

Why it matters

This incident highlights the fragility of memory safety in high-performance LLM inference engines. As providers push for larger context windows and faster throughput, they often adopt complex architectures like disaggregated prefill and decode. These systems rely on precise state management across multiple nodes. When configuration values such as context length and memory pool size are not strictly enforced, edge cases can lead to severe security vulnerabilities. For engineering teams building on top of these platforms, it underscores the importance of understanding that performance optimizations like non-zeroed memory reuse come with significant responsibility for the infrastructure provider.

For developers using Workers AI or similar services, this event serves as a reminder that tenant isolation is not just a network-level concern but also a memory-management challenge within the inference engine itself. While Cloudflare’s rapid response limited the exposure, the fact that 155 users received leaked data indicates that even brief windows of vulnerability can have tangible consequences. It also demonstrates the value of coordinated disclosure and open-source collaboration, as the fixes developed by Cloudflare were merged upstream to benefit the broader SGLang community.

What you can do

• Update SGLang: If you operate your own inference infrastructure using SGLang, upgrade to version v0.5.22 or later immediately to incorporate the latest security patches.
• Validate Configurations: Ensure that your KV cache pool size is strictly larger than the maximum context length for every model you deploy. Do not rely on the engine to gracefully handle mismatches.
• Monitor for Anomalies: Implement logging and monitoring for response times and token counts. Responses that arrive unexpectedly fast for long prompts may indicate truncated processing or cache errors.
• Review Disaggregated Setups: If using disaggregated prefill/decode architectures, verify that rejection paths are handled correctly across nodes and that rejected requests do not proceed to KV transfer.
• Check Upstream Issues: Regularly review the issue trackers of the inference engines you depend on, such as SGLang, for known memory safety bugs and race conditions.
• Audit Long-Context Usage: If your application relies on very long context windows, perform specific integration tests that validate output coherence and isolation under load, rather than just checking HTTP status codes.