Everyone involved in AI should have read The Library of Babel [1].
It's about many things, but perhaps the most relevant idea here is that no information matters without understanding. We could generate all possible knowledge, but unless someone--a human--can verify and understand it, it doesn't count. The cure for mortality could be written on the moon, but if no one reads it, it hasn't really been discovered.
[1] https://maskofreason.wordpress.com/wp-content/uploads/2011/0...
While I get the point and it's true.
I'm a little more flexible, if the new knowledge (that human's don't understand) can be put into a mechanism and have an observable effect, I'd be happy enough. e.g. a new type of rocket fuel that burns 1000x more efficiently.
> but unless someone--a human--can verify and understand it, it doesn't count
Does it need to be a human or can it be some other form of life?
Update:
"I own nothing, have no privacy, [never have to think, and am not required to solve any problems,] and life has never been better."
https://en.wikipedia.org/wiki/You'll_own_nothing_and_be_happ...
> unless someone--a human--can verify and understand it, it doesn't count.
One counter-point: Humans have not figured out how general anesthesia works, but we use it every day to great effect.
https://en.wikipedia.org/wiki/Theories_of_general_anaestheti...