alt Hacker NewsI'd hope most functional adults understand that the Fields Medal and basically every other annual "prize" out there is awarded to both "recombinant" innovations and "new-dimensional thinking" innovations. Humans aren't going to come up with "new-dimensional" innovations in every field, every single year.
I'd say yes, LLMs "just" recombine things. I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus. (I'm open to being proven wrong.) But stuff like this is exactly the type of innovation LLMs are great at, and that doesn't discount the need for humans to also be good at "recombinant" innovation. We still seem to be able to do a lot that they cannot in terms of synthesizing new ideas.
Replies
I agree with almost all of what you have stated, save for a minor nitpick: I frankly don't think most functional adults think about the Fields Medal, similar annual prizes, or the qualities of the innovations of their candidate pools. I also think that that's totally okay. I think among a certain learned cohort of adults it's okay to hope that, and I think it's okay to imagine an idealized world where having an opinion on this sort of matter is a baseline, but I don't think it's realistic or fair to imply that (what I believe handwavily to be a majority of) adults are nonfunctional for not sharing this understanding.
I think an LLM trained on pre-calculus material would easily stumble into reinventing at least early calculus. It's already pretty easy for students to stumble into calculus from solid enough fundamentals.
We even think that the Babylonian astronomers figured out they could integrate over velocity to predict the position of Jupiter.
> I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus.
Yes but that is because there was not enough text available to create an intelligent LLM to begin with.
To keep my usual rant short: I think you’re assuming a categorical distinction between those two types of innovations that just doesn’t exist. Calculus certainly required some fundamental paradigm shifts, but there’s a reason that they didn’t have to make up many words wholesale to explain it!
Also we shouldn’t be thinking about what LLMs are good at, but rather what any computer ever might be good at. LLMs are already only one (essential!) part of the system that produced this result, and we’ve only had them for 3 years.
Also also this is a tiny nitpick but: the fields medal is every 4 years, AFAIR. For that exact reason, probably!
[flagged]
> I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus. (I'm open to being proven wrong.)
The experiment is feasible. If it were performed and produced a positive result, what would it imply/change about how you see LLMs?
Also, I'm not sure why CS people act like axioms are where you start. Finding them is very very difficult. It can take some real innovation because you're trying to get rid of things, not build on top of. True for a lot of science too. You don't just build up. You tear down. You translate. You go sideways. You zoom in. You zoom out. There are so many tools at your disposal. There's so much math that has no algorithmic process to it. If you think it all is, your image is too ideal (pun(s) intended).
But at the same time I get it, it is a level of math (and science) people never even come into contact with. People think they're good at math because they can do calculus. You're leagues ahead of most others around you, yes, and be proud of that. But don't let that distance deceive you into believing you're anywhere near the experts. There's true for much more than just math, but it's easy to demonstrate to people that they don't understand math. Granted, most people don't want to learn, which is perfectly okay too