logoalt Hacker News

johnnyApplePRNGtoday at 3:05 AM1 replyview on HN

Not surprised to see Claude significantly higher in scientific intelligence than Sol.

You can tell that Claude really does grasp a wide array of highly specific scientific and mathematical nuances... where's codex is just basically for coding and that's it.

That's the feel I get from the both of them anyways and I've used both on the 20x plan for the past week at length.

Don't get me wrong, codex is great at finding bugs and building games. It's great.


Replies

saithoundtoday at 5:18 AM

> You can tell that Claude really does grasp a wide array of highly specific scientific and mathematical nuances... where's codex is just basically for coding and that's it.

Can you elaborate or give some examples of mathematical nuances?

I am evaluating Sol and Fable on a fairly large dataset of subtly flawed informal mathematical arguments (task is to name propositions with incorrect proofs), and Sol is saturating the benchmark, while Fable is below 50% even with the most generous grading.

I don't work in the natural sciences, so I suspect you mean something different by "mathematical nuance".