A config gap you could see in the logs leaked 155 AI responses

The warning was printed at startup

On October 26, Cloudflare published its report on a tenant isolation failure in the GLM-5.3 model on Workers AI. Between August 27 and September 9, 155 responses to prompts longer than 681,280 tokens contained data from other customers' requests. Cloudflare says the timing and volume of affected requests are consistent with ordinary long-context use, and it found no evidence of deliberate exploitation. Security researcher Gadi Evron of Knostic reported the problem on September 8; Cloudflare took the model offline within 40 minutes of declaring an incident and restored it on September 9.

The detail worth sitting with: Cloudflare says it knew about the mismatch before launch. The model was configured for a 1,048,576-token context window, but the memory pool on its prefill GPUs held only 681,280 tokens. The engine prints that pool size at startup and exposes it as a metric. The team assumed oversized prompts would simply return an error.

How a shared memory pool leaks

When a language model reads your prompt, it stores intermediate results, the KV cache, in GPU memory. That memory is a shared pool of fixed-size pages. A request is assigned pages when admitted and releases them when finished. Cloudflare says pages are not zeroed between requests, which is common among inference engines because wiping them costs performance. Instead, the engine must track who owns each page.

Cloudflare serves GLM-5.3 in a split design: prefill GPUs process the prompt and build the cache, then hand it to separate decode GPUs that generate the reply. The decode side reserves pages based on the prompt's length.

Here the SGLang inference engine failed. A prompt too big for the pool was marked rejected and swapped for a one-token placeholder. But the admission checks in disaggregated mode asked whether a request was finished, not whether it was rejected, so the rejected request sailed through. Prefill wrote one page and signaled completion. Decode, which had reserved eight pages in the source's example, assumed all eight were freshly written and attended over the other seven, still holding earlier users' data. The client received an HTTP 200 and a coherent answer.

Why testing missed it

Cloudflare says its pre-launch long-context tests checked HTTP status codes and performance, not whether the output made sense. Successful codes looked like proof the engine handled the gap. This is the signal: the failure produced no error, no crash, and plausible text. The only symptom was that replies arrived faster than a full prefill should take.

It was also a known bug upstream. Another SGLang user had reported it as issue #30233 on July 6, before the leak window opened, and Cloudflare says it had been flagging similar issues since v0.5.10. The fix shipped in v0.5.20, with follow-ups in v0.5.21 and v0.5.22; Cloudflare recommends operators update to v0.5.22 or later.

The larger pattern

Multi-tenant inference now depends on open-source engines moving faster than their safety checks. Pool size is set at startup from leftover GPU memory, while context length is a separate setting, and, in Cloudflare's words, nothing forces the two to agree. Ownership of memory pages is shared across machines connected by RDMA, the direct memory transfer between nodes. Cloudflare says only GLM-5.3 was affected, because its other disaggregated deployments have pools larger than their context windows. That is a configuration margin, not an architectural guarantee, and Cloudflare itself calls relying on pool size alone fragile.

Questions You Should Be Asking

• Does your inference provider verify, from the running engine, that cache capacity covers the advertised context length before every deployment change?
• Do its pre-launch tests inspect model output for coherence and foreign content, or only status codes and latency?
• When an upstream engine has an open bug report touching isolation, what is the process for escalating it before launch?
• If your prompts exceeded the 681,280-token threshold on this model between August 27 and September 9, has the provider confirmed whether your data was among the 155 responses?
• Which of your own self-hosted engines run in disaggregated mode, and are they on a patched version?

What To Watch Next

Watch whether other inference providers and SGLang users publish their own pool-versus-context audits, and whether the upstream follow-ups Cloudflare references, including #38704, land in releases. The tell will be whether capacity checks become a standard deployment gate across providers or remain one company's post-incident fix.