alt Hacker NewsI build ai systems for ops automation and don’t understand the author’s pessimism.
I agree with that Evals are a scarce commodity rn. A business needs to define what good looks like. This is a laborious, and sometimes politically controversial, process.
Given good Evals, frontier llms are a magical tool that can automate tasks and do them more accurately than human ops teams.
Exactly which llm to use depends on the mixture of speed, cost and quality of the output.
Jev makes a claim to expand some regions of the Pareto frontier. I look forward to testing if this is true.
There are many areas of work we can’t automate rn. We cannot create good Evals either because time horizons are too long, or it’s too difficult to create good Evals.
That doesn’t mean there’s anything wrong with building good ai engineering systems in areas where it works magically.
> That doesn’t mean there’s anything wrong with building good ai engineering systems in areas where it works magically.
Without the evals, how do you know it works magically? I think that’s the entire point of the authors article, people are using AI tools, and trusting the AI own evaluation of its performance, without making any attempt to validate if the AIs confidence actually correlates to anything meaningful.
From what I’ve seen, the result tends to be very exciting demos, followed by incredibly mediocre results when deployed into a real environment. The lack of critical evaluation then provides a double whammy, because it takes longer than it should to realise the system isn’t actually working correctly, and then the team lacks the skills and understanding needed to improve the system they’ve built. You just kinda have to hope that the next model release is going to magically result in better performance for your particular scenario, which won’t be able to measure due to the lack of evals.