Google: Not Trying 404 Pages Isn't A Sign Of Crawl Issues

May 12, 2017 - 7:45 am 2 by
Filed Under Google News

google tired cat

The other day I reported that Google Crawls 404s After Other Pages So No Crawl Budget Is Sacrificed. So I asked, somewhat in a joking way "so maybe a spike in 404s mean you have some excellent crawl budget expenditure for your site?"

In any event, it seems others asked the question also where David Iwanow asked, if Google is not trying your 404s every now and then, is that a sign of a crawl issue. John Mueller from Google said no, it is not. Here are the tweets:

So don't freak out if Google isn't trying your 404s, it isn't a bad thing.

Dawn Anderson then reminded me on Twitter that John addressed this topic a year ago. In a video at around the 27 minute mark he went into this. Then at 30:49 into the video John said:

Yeah, I mean usually we use this more to kind of fill up the extra capacity that we have.

We say, well, we've crawled all the important URLs from this website. We know we could crawl 10,000 more URLs. So let's just double check the rest of the stuff on our list to make sure that we're not missing anything.

So it's not that you'd lose anything by us not crawling those URLs or you wouldn't gain anything by us not crawling those URLs. It's essentially just our system saying, well, there's this big backlog over here that we don't really know what to do with, so let's just double check it when we have extra time.

If Google has extra capacity, they will go at it. If not, not.

Forum discussion at Twitter.


Popular Categories

The Pulse of the search community


Search Video Recaps

Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: May 27, 2024

May 27, 2024 - 10:00 am

In Face Of AI Overview Backlash, Google Updates Docs With How To Show Web Only Results & How To Give Feedback

May 27, 2024 - 7:51 am
Google Search Engine Optimization

Google's John Mueller Blasts The Concept Of Toxic Links, Again

May 27, 2024 - 7:41 am
Google Search Engine Optimization

Some Reporting Fewer Links Reported In Google Search Console

May 27, 2024 - 7:31 am

Google Images "See Exact Matches" Helps You Find Who Stole Your Images

May 27, 2024 - 7:21 am

Google Mobile Tests Large Blue Visit Button On Search Results

May 27, 2024 - 7:11 am
Previous Story: Google Merchant Center Bug: Unexpected Image Processing Errors