It cant "invent new things". It can only reuse what it already knows about.

Thats why the math breakthrough a few weeks ago was so hotly debated. Because OpenAI is desperate to demonstrate that AI isnt just a fancy regurgitation machine, but it can actually develop novel thought. Because that would be the stock price jumps to end all stock jumps.

But then it turned out it was really just listening in on a math professors supposed-to-be-private conversations with another instance of openai, and it used his novel work as the trigger to prove the breakthrough first.

The reason people conclude AI 'thinks' is because tt can reference obscure or poorly documented things quickly (which is its primary advantage along with processing natural language prompts into tasks), which is why a lot of people with emotions confuse that action with inventing things.

Because the hard part is getting the simulation to be very accurate such that if you design something that works in your simulator, it will actually work in real life. And I would be extremely surprised if any LLM can actually do that correctly.

Getting an LLM to design something in its own simulator that is not accurate w.r.t reality is not useful nor terribly impressive.

> For the benefit of a layman, can you explain why this is so much different than a human doing it?

More broadly than the existing answers (which are correct), For a layman, I'd also add that LLMs are essentially 'brains in a vat'. They can't confirm ground truth about physical reality. They only know what's in their training data and prompt, which is incomplete and can be incorrect. Even with real-time external sensors they are limited to the sensor's margin of error, range and trusting it's working correctly.

When properly trained, fine-tuned and prompted, LLMs can be very effective in well-defined, non-physical domains like logic, writing, math and code but making things function in the real-world quickly spirals into combinatorial complexity.