This feels like a stupid idea, but it's not my tax money being wasted on it (yet).

An AI being used for offensive purposes would be so easy to counter due to Brandolini's Law. A much simpler AI could probably easily create dumb honey pot systems that looked like real systems with holes in them to lead more advanced and expensive systems on a wild goose chase.

Suddenly an AI creating security vulnerabilities is a core part of a future cyber defence strategy. Surely someone must be doing that already