> models being out there that enable hacking

It's been really nice to be able to breathe new life into some old hardware I had kicking around that's been discontinued. Some of it just needed old exploits applied to it, some of it needed some binary reverse engineering. Lots of security tools are dual-use as well; a model that knows nothing about hacking is going to have a really hard to time helping defend a system that's getting hacked (see the HuggingFace incident where HF staff were unable to use ChatGPT or Claude to analyze the logs from the attack because they hit security research guardrails).

> manufacturing viruses at a scale...

I'm not sure if you're referring to software viruses or biological viruses here.

If you mean software viruses, see above.

If you mean biological viruses, a lot of this gets back into the dual-use nature as well. I brew beer and mead. This involves cultivating specific strains of bacteria and providing them with a medium where they can convert sugar into CO2 and Ethanol. I don't think there's a good way to thin-slice the training data so that it can be an expert helper at Saccharomyces cerevisiae cultivation while being completely naive to Clostridium botulinum cultivation. Even further, it seems that helping someone make sure that they're not inadvertently cultivating Botulinum (e.g. water bath canning with insufficient pH) is useful.