alt Hacker NewsNew proposal for LLM-on-LLM wars: Have an automated system that retrieves every front page URL link on HN and feeds it into a "smart" LLM for analysis of whether or not the site html, css, JS and general template layout match that of a known LLM generation engine. Then give it a "probably vibe coded slop" rank number. If you can't beat 'em, let them fight...
Replies
Please, do not use LLMs to do actual statistics. They suck at doing actual statistics (like they suck at everything else).
Use proven traditional statistical methods like supervised learning for this. You can use traditional (not large) language models to tokenize the content and then a supervised learning trained on the most popular models to detect if those models generated the content.
I haven't tried it personally but my understanding from friends is that Pangram is very accurate, at least for now.
https://www.pangram.com/