alt Hacker News"We deployed GLM 5.3 with a 1,048,576-token context window, but the KV cache pool on our prefill GPUs only had room for 681,280 tokens. We were aware of this mismatch before launch. "
Did they advertise it as 1,048,576 token when they only had room for 681,280 tokens?
Replies
weird-eye-issue • today at 1:05 AM
Next sentence: "We expected the engine to return an error for any prompt it could not process, and our long-context tests returned successful responses, so we did not believe this to be an issue."
➕ show 4 replies
I read that as you can only do a single prefill up to 681k tokens, separate from cache hit or generation. I think people try pretty hard to avoid a cache miss on sessions that long and rarely do standalone requests that big so the situation triggering this condition is rare. If you run a normal coding session with many turns it would get to 1M ok.