logoalt Hacker News

bombcaryesterday at 9:06 PM1 replyview on HN

Wayback machine removes archives upon request, so there’s definitely stuff they don’t make publicly available (they may still have it).


Replies

super256today at 12:02 AM

You don't even need to do requests if you are the owner of the URL. Robot.txt changes are applied in retrospect, which means you can disallow crawls to /abc, request a re-crawl, and all snapshots from the past which match this new rule will be removed.