Ask Jeeves to Add a Parameter for "Hard Crawl"

Feb 10, 2005 - 10:18 am 1 by
Filed Under Ask.com

A thread at WebmasterWorld has a post where a member said he spoke with an Ask Jeeves engineer and they told him that they might be adding a parameter to tell the teoma spiders how "hard" to crawl. Now this is supposedly different then the Yahoo! Crawl Delay where, "You can add a "Crawl-delay: xx" instruction, where "xx" is the minimum delay in seconds between successive crawler accesses. If the crawler rate is a problem for your server, you can set the delay up to 60 or 300 or whatever value is comfortable for your server."

I am trying to find out more information about the validity of this WebmasterWorld thread and what it really means. In the thread, the member said the Ask Jeeves representative said that this is not a Crawl Delay. At this point, I do not see how a hard crawl would be anything else but a time delay.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: July 10, 2025

Jul 10, 2025 - 10:00 am
Google Updates

Google June 2025 Core Update Leading To Some Partial HCU Recoveries

Jul 10, 2025 - 7:51 am
Google Search Engine Optimization

Google Search Console Performance Report Average Position Bug?

Jul 10, 2025 - 7:41 am
Google Ads

Google Search Ads People Also Consider Carousel Format

Jul 10, 2025 - 7:31 am
Google Maps

Google Business Profiles Drops Utility Bill As Evidence For Appeal

Jul 10, 2025 - 7:21 am
Google

Google AI Mode Comes To Circle to Search and Lens

Jul 10, 2025 - 7:11 am
Previous Story: Major Yahoo Update Reported