Yes you are definitely onto something! Love to see more people talking about deterministic games.

My game is strictly deterministic, so I get bot movement for free - but the player has agency so I need to capture their deviations

That’s the tricky part! Right now I do capture input (actually just deviations) and can replay whole games, but I think I’m at the limits in terms of compression - talking bytes here not KB