"You should have a sub-hammer to adversarially enforce that your primary hammer accurately drives nails into wood"
We wouldn't accept such behavior from any other tool, machine, or computer program. At least most of us would not. To paraphrase the old saying: Some people, when confronted with a problem with an AI model think "I know, I'll use an additional AI model." Now they have two problems.
I find it very intriguing that two lineages of models -- from two different companies -- that are supposed to be painstakingly RL'd to become human-level programmers are actually consistently bad at it out of the box in very similar ways. One would think that at least one of OpenAI and Anthropic would (could) have pushed their model to a direction different from "if you can solve a 100-line problem in 10,000 lines, do it".
"You should have a sub-hammer to adversarially enforce that your primary hammer accurately drives nails into wood"
We wouldn't accept such behavior from any other tool, machine, or computer program. At least most of us would not. To paraphrase the old saying: Some people, when confronted with a problem with an AI model think "I know, I'll use an additional AI model." Now they have two problems.
I find it very intriguing that two lineages of models -- from two different companies -- that are supposed to be painstakingly RL'd to become human-level programmers are actually consistently bad at it out of the box in very similar ways. One would think that at least one of OpenAI and Anthropic would (could) have pushed their model to a direction different from "if you can solve a 100-line problem in 10,000 lines, do it".