To clarify my parent comment, I think it was an interesting experiment and seems like it was done well, and it may well be informative about what various frontier LLMs could do with recorded or world model footage.
My only point is to say this sort of experiment is where it ends. Neither Anthropic nor OpenAI will be coming out with a "drive your car from the cloud" subscription until we have FTL communication, meaning never.
What about Waymo's remote controlled cars? Why is this not an issue?
Is it plausible they can use the large GPT model, to distill a smaller car driving model only from it and then run that onboard. Seems like that will solve all your issues.