alt Hacker NewsCould a network be trained to transform physics state directly into the latent state and back?
Having a direct transformation would enable some interesting experiments.
How is the latent state different when everything else stays the same, but you change one physics value, like player one velocity? Is there a cyclical pattern of activation that correlates strongly with the seconds digit of the clock? Can you decode the latent state, give players full boost, and then re-encode it for infinite boost, without losing continuity?
Edit: There sure are a lot of papers on interpretability.
Would be a great idea to see how much we could manipulate the latent space and whether it has some internal structure w.r.t the physical state. I guess the only unknown is how the world model would show robustness to latent states that are transformed through this network