WordPress System Admin Calls Out Google's Evil Spiders

Jun 23, 2010 - 8:20 am 2 by

A Google Webmaster Help thread has one of the system administrators from Automattic, the company behind the most popular blogging platform out there, WordPress, complaining about GoogleBot's recent behavior.

He said recent GoogleBot behavior is similar to that of a DoS attack. He explained that "starting around 6:57 EDT today, one of our user's site started getting hammered by dozens if Google Bots instances concurrently, coming from dozens of legitimate IPs in the 66.249.71.n range."

He verified it was from Google's IP range and is concerned that there is a "bug" with Google's crawlers. He then warns Google that WordPress may need to block Google from crawling the blogs if they don't act fast. He said:

We of course do not want to block the Google bot from crwaling this or any of the public content on any of the WordPress.com sites, but since it does not support any of the crawl throttling directives others do (http://www.google.com/support/forum/p/Webmasters/thread?tid=2caa38cb3d55bfa0&hl=en), I'm at a bit of a loss on how to effectively stop this from happening again without taking some drastic measures I'd like to avoid.

JohnMu of Google replied pretty quickly saying:

Thanks for posting -- I'll pass it on to the team here to review. If this is just for one site, using Webmaster Tools would probably be the best way to solve this problem, but it might be useful for our engineers to take a look to see what caused it in the first place.

We have not heard back from Google if it was a bug or if anything changed. Nor did we hear from WordPress if it has been resolved.

But it is pretty cool that WordPress posted this.

Forum discussion at Google Webmaster Help.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Updates

Google Urges Patience As The March 2024 Core Update Continues To Rollout

Mar 18, 2024 - 7:51 am
Google

Official: Google Replaces Perspective Filter With Forums Filter

Mar 18, 2024 - 7:41 am
Google Maps

Google Business Profiles Now Offers Additional Review After Appeal Is Denied

Mar 18, 2024 - 7:31 am
Google Maps

EU Searchers Complaining About Google Maps Features Changes Related To DMA

Mar 18, 2024 - 7:21 am
Google

Google Showing Fewer Sitelinks Within Search

Mar 18, 2024 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: March 15, 2024

Mar 15, 2024 - 4:00 pm
Previous Story: Google's Maile Ohye On The MayDay Update