logoalt Hacker News

felipeeriastoday at 4:02 AM1 replyview on HN

Each person writes in a different personal way, so writing “like a human” would actually require a model being able to purposefully make the specific choices that an individual human writer does.

However, general purpose LLMs like Fable have been trained on huge amounts of all kinds of data, and therefore find it exceedingly hard to break out of the grooves carved by that data. They can’t avoid defaulting to centroids and averages, even when they are trying not to. This makes it possible for classifiers like Pangram to discriminate their writing.

A plausible way to work around this limitation would be to train a LLM on a limited and cohesive subset of writing materials, so it would absorb their specific writing style.

One example might be Talkie, a LLM trained on pre-1930’s English text. Talkie is a far smaller and less powerful model than Fable.

And yet, Talkie’s writing is so distinctive that it is often classified as human by Pangram.


Replies

mr_mitmtoday at 7:53 AM

Is pangram using knowledge about specific models? Could you throw it off by having several models generate a paragraph each or so?