The effective altruism/rationalism/AI xrisk people have always had a shockingly poor grasp on subjects outside computer science, despite their attempts to speak on them. I don't blame the actual biologists and chemists working at the frontier labs for wanting to skim a few bucks off all the money flying around, though! I know a couple who've had not-so-kind words to say about their employers' intelligence.

I suspect there's at least some "telling the bosses what they want to hear" going on. A massive financial incentive exists to exaggerate and fearmonger even internally to the company, because it makes you and your job seem more important.

Half or more of Hacker News is constantly pushing the idea that frontier LLMs are stochastic parrots and basically useless, even in the face of the Hugging Face incident which most people also would have described as movie plot fiction until it happened. I expect biologists are even less well versed in the abilities of frontier models.

What's more, you can just try a jailbreak on a model yourself to see just how much detailed, step-by-step direction you can get to build bio-terror materials.