WordPress System Admin Calls Out Google's Evil Spiders

Jun 23, 2010 - 8:20 am 2 by

A Google Webmaster Help thread has one of the system administrators from Automattic, the company behind the most popular blogging platform out there, WordPress, complaining about GoogleBot's recent behavior.

He said recent GoogleBot behavior is similar to that of a DoS attack. He explained that "starting around 6:57 EDT today, one of our user's site started getting hammered by dozens if Google Bots instances concurrently, coming from dozens of legitimate IPs in the 66.249.71.n range."

He verified it was from Google's IP range and is concerned that there is a "bug" with Google's crawlers. He then warns Google that WordPress may need to block Google from crawling the blogs if they don't act fast. He said:

We of course do not want to block the Google bot from crwaling this or any of the public content on any of the WordPress.com sites, but since it does not support any of the crawl throttling directives others do (http://www.google.com/support/forum/p/Webmasters/thread?tid=2caa38cb3d55bfa0&hl=en), I'm at a bit of a loss on how to effectively stop this from happening again without taking some drastic measures I'd like to avoid.

JohnMu of Google replied pretty quickly saying:

Thanks for posting -- I'll pass it on to the team here to review. If this is just for one site, using Webmaster Tools would probably be the best way to solve this problem, but it might be useful for our engineers to take a look to see what caused it in the first place.

We have not heard back from Google if it was a bug or if anything changed. Nor did we hear from WordPress if it has been resolved.

But it is pretty cool that WordPress posted this.

Forum discussion at Google Webmaster Help.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Core Update Volatility, Helpful Content Update Gone, Dangerous Search Results & Ads Confusion - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: April 17, 2024

Apr 17, 2024 - 4:00 pm
Google Updates

Deepening Google Core Ranking Volatility Hits Yesterday & Today

Apr 17, 2024 - 8:01 am
Google Maps

Google Maps No Longer Will Support Draft Reviews

Apr 17, 2024 - 7:51 am
Google Search Engine Optimization

Google Search Console Adds Unused Ownership Tokens

Apr 17, 2024 - 7:41 am
Google Ads

New Google Ads AI Generated Image Tool For Demand Gen Campaigns

Apr 17, 2024 - 7:31 am

Google Tests Horizontal Lines For Sitelinks & People Also Ask

Apr 17, 2024 - 7:21 am
Previous Story: Google's Maile Ohye On The MayDay Update