Nobody on HN is a bigger AI stan than I am -- well, maybe that SimonW guy, I guess -- but the truth is that problems involving unit conversions are among the riskiest things you can ask an LLM to handle for you.
It's not hard to imagine why, as the embedding vectors for terms like pounds/kilograms and feet/yards/meters are not going to be far from each other. Extreme caution is called for.
That sounds like the sort of thing I'd expect them to be good at. What goes wrong?