WordPress System Admin Calls Out Google's Evil Spiders

Jun 23, 2010 - 8:20 am 2 by

A Google Webmaster Help thread has one of the system administrators from Automattic, the company behind the most popular blogging platform out there, WordPress, complaining about GoogleBot's recent behavior.

He said recent GoogleBot behavior is similar to that of a DoS attack. He explained that "starting around 6:57 EDT today, one of our user's site started getting hammered by dozens if Google Bots instances concurrently, coming from dozens of legitimate IPs in the 66.249.71.n range."

He verified it was from Google's IP range and is concerned that there is a "bug" with Google's crawlers. He then warns Google that WordPress may need to block Google from crawling the blogs if they don't act fast. He said:

We of course do not want to block the Google bot from crwaling this or any of the public content on any of the WordPress.com sites, but since it does not support any of the crawl throttling directives others do (http://www.google.com/support/forum/p/Webmasters/thread?tid=2caa38cb3d55bfa0&hl=en), I'm at a bit of a loss on how to effectively stop this from happening again without taking some drastic measures I'd like to avoid.

JohnMu of Google replied pretty quickly saying:

Thanks for posting -- I'll pass it on to the team here to review. If this is just for one site, using Webmaster Tools would probably be the best way to solve this problem, but it might be useful for our engineers to take a look to see what caused it in the first place.

We have not heard back from Google if it was a bug or if anything changed. Nor did we hear from WordPress if it has been resolved.

But it is pretty cool that WordPress posted this.

Forum discussion at Google Webmaster Help.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 27, 2024

Mar 27, 2024 - 4:00 pm
Google Ads

Google Local Service Ads Sends Email Asking You To Upload Photos

Mar 27, 2024 - 7:51 am
Google Maps

European Hotels Can Remove Pricing Information From Google Hotel Search

Mar 27, 2024 - 7:41 am
Google Search Engine Optimization

SEOs, Please Don't Remove Contact Us & About Us Pages

Mar 27, 2024 - 7:31 am
Misc Google

Google Search Developer Docs Gain AI Generated Help Features

Mar 27, 2024 - 7:21 am
Google Maps

Google Local Reviews Reactions Notice

Mar 27, 2024 - 7:11 am
Previous Story: Google's Maile Ohye On The MayDay Update