GoogleBot Completely Stop Crawling Yesterday For SOPA?

Jan 19, 2012 - 9:26 am 2 by

robot turtleDid you notice that your visits from GoogleBot, Google's web crawler, slowed to virtually almost nothing yesterday? The reason for that was because of the SOPA protests and Google slowed GoogleBot activity in order not to hurt those sites in their search results.

There are several threads where webmasters claim GoogleBot didn't crawl their sites at all yesterday. Google said they "slowed" the crawling behavior of GoogleBot but I guess in some cases, they stopped crawling for some sites completely. You have to assume they continued to crawl news sites?

A Google Webmaster Help thread and WebmasterWorld thread have webmasters and SEOs claiming they saw zero GoogleBot activity on their sites at all.

One webmaster said:

For some reason, googlebot is not accessing my site since 3 hours ago. Normally, I get thousands of googlebot visits per hour (and that ok for me), but since las 3 hours I'm not getting any googlebot visit.

Another said:

Well Googlebot seems to have all but disappeared on my main site. It may be on all sites then?

In response, Google's Pierre Far explained again:

We realize many webmasters are concerned about the medium-term effects of today's blackout. As a precaution, the crawl team at Google has configured Googlebot to crawl at a much lower rate for today only so that the Google results of websites participating in the blackout are less likely to be affected.

In any event, things should be back to normal today.

Forum discussion at Google Webmaster Help and WebmasterWorld.

Image credit to ShutterStock for robot turtle.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Core Update Volatility, Helpful Content Update Gone, Dangerous Search Results & Ads Confusion - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: April 12, 2024

Apr 12, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google Core Update Volatility, Helpful Content Update Gone, Dangerous Google Search Results & Google Ads Confusion

Apr 12, 2024 - 8:01 am
Google Search Engine Optimization

Google: Indexing & Algorithm Updates Are Independent

Apr 12, 2024 - 7:51 am
Google Search Engine Optimization

Google Structured Data Carousels Beta Docs Clarifies Feature Availability & Markup Location

Apr 12, 2024 - 7:41 am

Google: Ranking In Shopping, Images & Other Verticals Doesn't Hurt Your Web Rankings

Apr 12, 2024 - 7:31 am

Google Knowledge Panels - Mentioned People

Apr 12, 2024 - 7:21 am
Previous Story: AdWords Impression Share Metrics Algorithm Update & Ad Group Level