I regretfully suspect this actually is the future of UI. You just need smart local models running fast enough to keep the framerate high, but that’s inevitable in the long term, isn’t it?
Why would you do that instead of having the model do the task you're using the UI for? It could generate UI widgets on-the-fly as needed to take richer inputs.
What if we're really just living in a world model? How big of a GPU is required to run the universe I wonder?
This makes absolutely no sense what so ever.
There's a good reason why the UI for Excel, Photoshop and Word has hardly changed and remains consistent for years and that has worked for billions of people.
Many HN readers have forgotten the Windows 8 UI disaster already.
We had decades to work on making software more performant, but instead most users' space-age 2nm chips spend most of their time running Javascript because it was faster to make software that way.