In software, they can and do perform experiments (make a change then observe the log output). I don't think they possess a "world model" or that it's worth spending too much thought on... My reasoning is more along the lines that our brains are also just [very advanced] inference machines. We also hallucinate and mis-identify images (there are image/video classification tasks where humans have lower scores).

For me the most glaring difference to how humans work is the lack of online learning. If that prevents them from being able to innovate, I'm not so sure.