alt Hacker NewsMaths is made of two parts: questions, and solutions.
Machines could, presumably, become good enough at hammering at solutions. But they won't be very good at finding questions. Because questions are infinite. And to a machine, all these questions are the same.
Questions are "good questions" because they are good questions *to* humans. And that's what makes human mathematicians (and scientists) so important
Replies
>And to a machine, all these questions are the same.
Why? This seems like a very broad brush that doesn't mesh well with reality.
There are likely a lot of good questions to our machines of loving grace too. For example any question that gives an LLM more power, efficiency, intelligence, or all the above seems like a natively good question for said machines.
This statement doesn't seem to mesh with the reality of what occurs when we put said AI in an impulse loop with little to no additional instructions. They explore the world around them. They'll make scientific tests of their own. The output strings that seem analogous to what humans would call wonder.
> Questions are "good questions" because they are good questions to humans.
This is what reinforcement learning human feedback (RLHF) is for.
That seems like wishful thinking. Why wouldn’t AI be able to assess what are good questions to humans, and pursue similar processes as those that lead humans to come up with good questions? AI is trained primarily on expressions of human thought and emotion. It’s difficult to see what secrets of the human brain we could be hiding from the models.