GoogleBot Completely Stop Crawling Yesterday For SOPA?

Jan 19, 2012 • 9:26 am | comments (2) by twitter Google+ | Filed Under Google Search Engine Optimization

robot turtleDid you notice that your visits from GoogleBot, Google's web crawler, slowed to virtually almost nothing yesterday? The reason for that was because of the SOPA protests and Google slowed GoogleBot activity in order not to hurt those sites in their search results.

There are several threads where webmasters claim GoogleBot didn't crawl their sites at all yesterday. Google said they "slowed" the crawling behavior of GoogleBot but I guess in some cases, they stopped crawling for some sites completely. You have to assume they continued to crawl news sites?

A Google Webmaster Help thread and WebmasterWorld thread have webmasters and SEOs claiming they saw zero GoogleBot activity on their sites at all.

One webmaster said:

For some reason, googlebot is not accessing my site since 3 hours ago. Normally, I get thousands of googlebot visits per hour (and that ok for me), but since las 3 hours I'm not getting any googlebot visit.

Another said:

Well Googlebot seems to have all but disappeared on my main site. It may be on all sites then?

In response, Google's Pierre Far explained again:

We realize many webmasters are concerned about the medium-term effects of today's blackout. As a precaution, the crawl team at Google has configured Googlebot to crawl at a much lower rate for today only so that the Google results of websites participating in the blackout are less likely to be affected.

In any event, things should be back to normal today.

Forum discussion at Google Webmaster Help and WebmasterWorld.

Image credit to ShutterStock for robot turtle.

Previous story: AdWords Impression Share Metrics Algorithm Update & Ad Group Level


mobile application testing

01/20/2012 05:01 am

Quite Interesting Though! Seems that Google is also opposing SOPA act which is Good move! but slowed down the Google bot will definitely made a huge impact on these websites which have unique and qualitative content!

Warren Lowe

01/20/2012 10:55 am

That explains it. Very clever of Google to think of that. Unless it was prompted by an E-mail from one of those companies.

blog comments powered by Disqus