This is why we need the HTTP 402 standard to become common.

If websites charge pennies per AI crawl, they will make more money than ever being reference in that 6.2% of websites that get cited (of which even another small percent get any follow through that leads to a sale or ad click)

HTTP 402 also basically extends the pay per token model people have gotten used to with AI model providers, except applied to the whole web, with the added privacy benefit in that there is no need for sellers of content to “know their customer”, and indeed it may even be impossible to do so because of how the payment gateways operate.