Yeah, it's such a subjective feature. I don't really worry about the five minute timeout, but sometimes if I'm conscious about the fact that I have an 800k token context window that I haven't interacted with for a day, I will switch to Opus for compaction and then back to Fable for interaction.
In general, though, some kind of API ping on a timeout does not support my personal workflow, which involves dozens of active or stagnant agent sessions that stay open for weeks.