alt Hacker NewsYup. Recently built something that needs to do low volume scraping. About 40% success rate - rest hits bot detection even on first try
Yup. Recently built something that needs to do low volume scraping. About 40% success rate - rest hits bot detection even on first try
Did you have rate limits built in? Ultimately scraping tools will need to mimic humans. Ironic.
I wonder if bots/ai will need to build their own specialized internet for faster sharing of data, with human centered interfaces to human spaces.