Google Updates Webmaster Tools & Crawl Rate Controls

Dec 5, 2008 - 8:33 am 3 by

Google has made two changes to Webmaster Tools. The first is they consolidated your settings into a single page for a site and the second is they are now giving more control over your crawl rate.

There is a new "Settings" link on the left hand side of a site profile in webmaster tools. That settings section gives you the ability to manage a site's geographic target, preferred domain, enhanced image search preference and your crawl rate control. Note, some of these settings expire after 90-days. Here is a picture:

Google Webmaster Tools Settings

The second enhancement is the ability to allow Google to decide how to fast or slow crawl your site, or to give you those options. The screen capture pretty much explains it:

Google Crawl Rate

This is one of those settings that set back after 90-days. So keep checking it if you make a change.

There are some webmasters who are not allowed to change the setting. They are told Google has set their rate and they cannot do anything about it. There is discussion about this notification at Google Webmaster Help and DigitalPoint Forums. But why can't they change it? One suspects the site is hosted at Blogger or a free hosting site, but JohnMu of Google wrote:

You usually won't have to fiddle with those settings, which is why they aren't available to all sites. We usually work out how much we can (and want to) crawl automatically. I would generally only change these settings if we're crawling your site too hard (and slowing down your server).

Forum discussion at Google Webmaster Help and DigitalPoint Forums.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Gvolatility, Bing Generative Search, Reddit Blocks Bing, Sticky Cookies, AI Overview Ads & SearchGPT - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: July 26, 2024

Jul 26, 2024 - 10:00 am
Search Video Recaps

Google Volatility, Bing Generative Search, Reddit Blocks Bing, Sticky Cookies, AI Overview Ads & SearchGPT

Jul 26, 2024 - 8:01 am
Google

Google Gemini Adds Related Content & Verification Links

Jul 26, 2024 - 7:51 am
Other Search Engines

SearchGPT - OpenAI's AI Search Tool

Jul 26, 2024 - 7:41 am
Search Engine Optimization

Google's John Mueller: Don't Use LLMs For SEO Advice

Jul 26, 2024 - 7:31 am
Google

Google Search With Related Images Carousel Below Image Box

Jul 26, 2024 - 7:21 am
Previous Story: Google Webmaster Help Finds A New Home