logoalt Hacker News

teknetoday at 2:11 AM1 replyview on HN

Data point of one, but after a few months of uselessness, I have managed to get some pretty serious, measurable performance improvements with AI optimizations -- order-of-magnitude speedups of business critical processes which took days as well as significant latency reductions.

But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.


Replies

jeffbeetoday at 3:38 AM

What I want is the PMU in the loop.