It's going to get worse before it gets better:
- https://docs.sweeting.me/s/ai-scraping-will-force-us-towards...
- https://docs.sweeting.me/s/so-you-want-to-build-a-social-net...
I've worked at multiple AI browser companies at this point and I think the best way forward is to start paying for content again honestly. Brave and BAT were too early but we need more things like that with lower friction and more primitives baked into the browser. The AI bots that create value for people and have money will happily pay to get access to the content, keeping it alive in the process and paying for the maintenance cost of allowing good bots.
DBSC, DPoP, and Google's latest "Google Cloud Fraud Defense" reCAPTCHA using phone are another approach where you use hardware secure enclaves to prevent cookies from leaving devices and attest to 1 visitor = 1 physical device. I don't like these because I want my personal agents to be able to access the things I can on my behalf. If I misbehave then block me & bots, not all bots, this is not as impossible as companies make it sound.
The worst possible approach is to require biometric or government ID, that's the one we all need to push hard against, as it's bad for everyone involved.
> paying for content [...] The worst possible approach is to require biometric or government ID,
IMO a good middle-ground is a "proof of stake", where you just need to prove you're an entity that has some meaningful incentive to follow the rules and be nice, because you'll take some damage if the site tries to terminate the relationship.
That differs from other options like paying the site money (perverse incentive problem) or proving a relationship to a unique human (privacy dangers) or that you're a human and not a robot (difficult to test.)
Imagine a vending-machine at city-hall, where anyone can spend $5 to a legally-recognized charity of their choice. In exchange, they get some anonymous tokens [0] can be loaded into a web browser for use.
The point is we can all benefit by strategically relaxing requirements.
[0] Technical details of the anonymity and deterring resale etc. are important, but I'm gonna shelve them for the moment in the interests of space/brevity.