alt Hacker NewsTime to dump this test. Probably not a coincidence every version has the same rolling green hills, gradient blue sky, sun in the corner, etc.
Replies
ceroxylon • yesterday at 11:08 PM
I partially agree, but in this case it kinda illustrates that it may not be worth using Terra on any reasoning level below high; those are some awful penguins on bikes.
modriano • yesterday at 8:40 PM
I don't know. If they were training on this, I feel like they would be able to get the shape of a bike frame right; it's a pretty simple polygon, and a lot of the bike frames that are getting generated would be impossible to steer.
➕ show 1 reply
dbt00 • yesterday at 8:37 PM
Goodhart's law.
On the one hand: yes, pelicans on bikes are definitely in the training set at this point.
On the other hand: the test is clearly not saturated, given that you can see a clear difference in output at the various reasoning levels / model versions.