A WebmasterWorld thread asks how long does Google send out their crawlers to a page that has been removed (i.e. a 404 error not found status)?
The simple answer, as mentioned in the thread is, forever.
Google will continue to try to see if the 404ed page has ever been reinstated as a live page.
Google even provides these errors in Google Webmaster Central, where you can see web crawl errors including 404 not found errors. (note to Vanessa, fix the link on here for the 404 link to go to here).
Why does Google continue to try to access these pages? As g1smd said:
Say you bought a domain name from someone and put up a new site. Say that Google refused to pick up your /about.html and /contact.html pages, and that eventually you found that it was all because once a page went 404 Google refused to ever look at that URL again.You would think that was a bad policy. That is why it doesn't work like that.
Forum discussion at WebmasterWorld.
