logoalt Hacker News

ACCount37yesterday at 3:19 PM0 repliesview on HN

Constantly. Minor revisions can easily "wobble" on benchmarks that the training didn't explicitly push them for.

Whether it's genuine loss of capability or just measurement noise is typically unclear.