alt Hacker News"AI writing is vague and wrong in hard-to-notice ways" This one felt to me, at work I lost a lot of time re-reading and correcting AI prose that was launched to "summarize" a collective white paper, and where subtlety and nuance disappeared in ways that harmed the initial ideas/arguments I was writing about, but in sneaky ways.
Replies
michaelchisari • today at 7:20 PM
Most of us have worked our whole lives to be writers in some domain: Code, documentation, specifications, product requirements, corporate communication, etc.
If we don't want to be writers, then we have to be editors. And editing is an entirely different job and it's not an easy one. In many ways it's harder.
Especially when LLMs love writing novels when all we need is a short story or less.
➕ show 1 reply
AI is very bad at properly handling statements that make heavy use of vague quantifiers (e.g. "some", "most") and also commits a lot of pretty serious logical fallacies. It is also bad at handling subtle logical negation, generally.
One of the most egregious negation issues I run into a lot is when I (or someone) makes a statement of the form: "not X" or "X is thus not true", and the AI then proceeds to interpret or summarize this as 'whatever is the opposite of X is the case'". This will cause it to go down a useless path investigating or disputing the opposite of X, which generally has no relevance or bearing on anything.
It also often very harmfully will replace your carefully chosen words with weirdly specific academic operationalizations or formalisms, then again waste huge amounts of text refuting / showing "problems" that result from that formalism, all of which again have no bearing or relevance on the original statement. An example would be you saying something like "intelligence, generally, must surely explain some of the differences in X", and then it will go "actually IQ does not correlate with X", unless you specifically tell it not to conflate psychometric IQ with intelligence generally.
Sometimes this is helpful, but the more specific / technical the domain, the more often you specifically have to prevent it from going down stupid paths that should be obvious given the expert context and wording, because it can seem almost hungry to try to catch you in some kind of insipid 'gotcha'. Much of these issues often clearly arise immediately from the first-pass "reword what the user said" part, given the reasoning traces.