alt Hacker NewsI think we are about to enter a world where security compromises occur at least an order of magnitude more often due to AI. The fact that AI is sometimes wrong benefits the attacker because they can just automate attacking with agents until they breach a system and the failed attempts don't matter.
Nicholas Carlini has a good talk about the risks on his homepage https://nicholas.carlini.com/ (can't link to the talk directly for some reason)
The corollary is that service access will become increasingly penalized, authenticated, and finicky as the most general first line defense against automated "informed brute force" attackers will be to slow or ban access that looks like them.
If you've got a lot of roving bandits with fancy lockpicks, you're village needs to harden the walls, reduce the number of entry points, identify and document entrants more universally, check under more carriages and inside more trunks for stowaways, etc. And with all that comes delays, imprivacy, accusations, false positives.
The low trust world that we're headed towards isn't going to be nearly as fun.