alt Hacker NewsOP here. I liked Macro-o1 (Marco1?) but as you pointed out, we need to teach these models to spend their system 2 thinking more economically.
OP here. I liked Macro-o1 (Marco1?) but as you pointed out, we need to teach these models to spend their system 2 thinking more economically.
One of the things complicating the example is that the counting-letters task is actually a legitimately hard one for it, even if it's trivial for us: it's bad at letter stuff because of the way it represents text with tokens. I believe the problem exists, but the letter thing isn't necessarily a great example of something that it should recognize as trivial.