WordPress System Admin Calls Out Google's Evil Spiders

Jun 23, 2010 - 8:20 am 2 by

A Google Webmaster Help thread has one of the system administrators from Automattic, the company behind the most popular blogging platform out there, WordPress, complaining about GoogleBot's recent behavior.

He said recent GoogleBot behavior is similar to that of a DoS attack. He explained that "starting around 6:57 EDT today, one of our user's site started getting hammered by dozens if Google Bots instances concurrently, coming from dozens of legitimate IPs in the 66.249.71.n range."

He verified it was from Google's IP range and is concerned that there is a "bug" with Google's crawlers. He then warns Google that WordPress may need to block Google from crawling the blogs if they don't act fast. He said:

We of course do not want to block the Google bot from crwaling this or any of the public content on any of the WordPress.com sites, but since it does not support any of the crawl throttling directives others do (http://www.google.com/support/forum/p/Webmasters/thread?tid=2caa38cb3d55bfa0&hl=en), I'm at a bit of a loss on how to effectively stop this from happening again without taking some drastic measures I'd like to avoid.

JohnMu of Google replied pretty quickly saying:

Thanks for posting -- I'll pass it on to the team here to review. If this is just for one site, using Webmaster Tools would probably be the best way to solve this problem, but it might be useful for our engineers to take a look to see what caused it in the first place.

We have not heard back from Google if it was a bug or if anything changed. Nor did we hear from WordPress if it has been resolved.

But it is pretty cool that WordPress posted this.

Forum discussion at Google Webmaster Help.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Search Console Difficulties, Google Ranking Volatility, Zero Click Study, Google Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: July 11, 2024

Jul 11, 2024 - 10:00 am
Google Ads

Google Ads Tests People Also Browsed Carousel

Jul 11, 2024 - 7:51 am
Google Ads

Google Ads: Double Row Carousel Shopping Search Ads

Jul 11, 2024 - 7:41 am
Google Maps

Google Business Profiles Consumer Alerts Document

Jul 11, 2024 - 7:31 am
Google Search Engine Optimization

Google Search Tests Thin Video Snippets

Jul 11, 2024 - 7:21 am
Google Search Engine Optimization

Google: SEO Feedback On The Last Core Update Was Really Helpful

Jul 11, 2024 - 7:11 am
Previous Story: Google's Maile Ohye On The MayDay Update