Search engines seem to index HN in near real time. They must have custom scraping code to follow the incrementing post IDs.