logoalt Hacker News

epolanskiyesterday at 7:32 PM0 repliesview on HN

I built an LLM that has access to documentation before doing code reviews and forces devs to update it with each pr.

Needless to say, most see it as an annoyance not a benefit, me included.

It's not like it's useless but... people tend to hate reviewing LLM output, especially on something like docs that requires proper review (nope, an article and a product are different, an order and a delivery note are as well, and those are the most obvious..).

Code can be subpar or even gross but to the job, but docs cannot be subpar as they compound confusion.

I've even built a glossary to make sure the correct terms are used and kinda forced, but LLMs getting 95% right are less useful than getting 0, as the 5% tends to be more difficult to spot and tends to compound inaccuracies over time.

It's difficult, it really is, there's everything involved from behaviour to processes to human psychology to LLM instructing and tuning, those are difficult problems to solve unless your teams have budgets that allow you hiring a functional analyst that could double as a technical and business writer, and these figures are both rare and hard to sell to management. And then an LLM is hardly needed.