alt Hacker NewsNot sure I get that. If AI is not working for humans' benefits (i.e. it is not working towards an optimization problem set-up by humans), whom is it supposed to benefit, then? (i.e. how is that better than an entropy-producing machine)?
Same for 2, there are so many infinite ways to boil the oceans, but so few oceans to boil to begin with. Better make sure that this insane energy (both in the physical, due to natural resources scarcity, as well as intellectual) is spent towards meaningful and useful ends. We can no longer be the judges of that if we can't comprehend what we got in return.
Replies
>> If AI is not working for humans' benefits
>> not working towards an optimization problem set-up by humans
I am suggesting neither of the two.
Not a great analogy but a parent may be working for an infant's benefit without the infant yet being able to understand. Taking an arbitrarily broad example, the problem to optimize for could be "help humanity advance", and other aspects could be subgoals of the same including what mathematics is useful.
It's not clear current what the human benefit of current mathematics is. Can people figure out what's important?
It can still benefit humans, but there are plenty of situations where humans prioritize guarantees of formal correctness over the ease of navigating granular details.