GoogleBot Crawling Less or More Efficiently?

Sep 2, 2010 • 8:21 am | comments (4) by twitter Google+ | Filed Under Google Search Engine Optimization

The obsessed (in a cool geeky way) webmasters at WebmasterWorld are discussing recent patterns they noticed with how Google's spider, GoogleBot has been accessing their site recently.

One said GoogleBot is crawling 50% less than it normally does. Others are saying they are just seeing GoogleBot crawl differently, but not less, per say. In addition, another person said Google stop crawling almost completely, but that is likely another issue.

Tedster explained what he has recently seen:

The sites I work with have been showing a changed crawling pattern since June, but it's not really a reduced crawl. On average more pages are being spidered over any extended period, but the whole pattern contains more spikes - occurring on about a 7-8 day cycle.

Have you noticed new or different GoogleBot crawling behavior?

We did report GoogleBot crawling from multiple locations at the same time, them having new crawling issues and acting all weird recently. Remember, this may all be caffeine related, so keep that in mind.

Forum discussion at WebmasterWorld.

Previous story: Logged In/Out: Google Keyword Tool Gives Different Results



09/02/2010 01:55 pm

We have noticed a change on the way google is crawling our client's sites. It may be more accurate, and it hasn't affected our overall rankings, but it is less often.

Michael Martinez

09/02/2010 05:49 pm

I agree. They don't seem to be crawling as much as I am used to. But I've seen periods like this in the past. It's too early to be calling the shot.


09/02/2010 08:05 pm

they are probably just doing some housekeeping before crawling again. i wouldn't worry about it.

Carla Ackley

09/05/2010 11:44 pm

Same here. Noticing client sites getting crawled less, and less pages getting indexed. Will have to see how this works over time. Seeing this for a couple of months now. I had a dream about it, that the "Google Robots" were malfunctioning, so I was noticing this subconsciously. Carla Ackley, President Ackley's Custom Site Submission Search Engine Optimization since 1994 Internet Marketing Services Web Redesign & Webmaster Services Ask for a Free Site Review 800-969-9598 - toll free 623-587-9499 - voice 623-321-0208 - fax Skype: classichits1 - email

blog comments powered by Disqus