It's hard for me to imagine a stateless operation as intelligence per se

That's an interesting point. My take would be to say that we shouldn't think of the AI as being just the model, but should include the harness. At that level, clearly we can keep state / context and that is probably a more natural mapping to our intuitive understanding of "intelligence".