Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world.
In this view, we are essentially living inside a high-fidelity generative model. Our brains are constantly 'hallucinating' a predicted reality based on past experience and current goals. The data from our senses isn't the source of the image; it's the error signal used to calibrate that internal model. Much like Genie 3 uses latent actions and frames to predict the next state of a world, our brains use 'Active Inference' to minimize the gap between what we expect and what we experience.
It suggests that our sense of 'reality' isn't a direct recording of the world, but a highly optimized, interactive simulation that is continuously 'regularized' by the photons hitting our retinas.
A kurzgesagt on this: Why Your Brain Blinds You For 2 Hours Every Day https://youtu.be/wo_e0EvEZn8 and the sources for that video - https://sites.google.com/view/sources-reality-is-not-real/
Like, "Your Brain Hallucinates Your Conscious Reality" as exposed by Anil Seth[1]? Found that one while searching for something like "the illusion of the self" a few years ago.
It’s also easy to find this treated in various philosophy/religion through time and space. And anyway as consciousness is eager to project whatever looks like a possible fit, elements of suggesting prior arts can be inferred back as far as traces can be found.
Another analogy that kinda fits in with what you're saying is the post-processing on smartphone "photos."
At what point does the processing become so strong that it's less a photograph and more a work of computational impressionism?
Also check out The Case Against Reality by Donald Hoffman
I think this is pretty well established as far as neurologists are concerned and explains a lot of things. Like dreaming for instance.. just something like the model running without sensory input constraining it.