Yahoo! Slurp on the Loose?

Jan 17, 2007 - 7:57 am 5 by
Filed Under Yahoo SEO

A WebmasterWorld and Search Engine Watch Forums threads are both reporting issues with Yahoo! Slurp (Yahoo!'s Crawler) indexing pages they should not be, and in quantities that may be harmful.

It appears that only specific bots are not obeying the robots.txt file and indexing pages are rates that can potentially cause server issues.

The specific IP addresses appear to be in the 74.6.x block. They do reverse DNS to inktomi, which is correct.

Forum discussion at WebmasterWorld and Search Engine Watch Forums.

 

Popular Categories

The Pulse of the search community

Google Search Volatility

More Details

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Ads

Google Ads Bidding Target Optimization Changing, Promotion Mode Beta, Smart Bidding Exploration Expands

Jun 15, 2026 - 8:00 am
Google

Google Search Rolls Out Information Agents In AI Mode For A Fee

Jun 15, 2026 - 7:51 am
Google Maps

Google Local Finder Interface Drops Pagination

Jun 15, 2026 - 7:41 am
Google Ads

Google Ads To Limit Ad Impressions From Unqualified Advertisers

Jun 15, 2026 - 7:31 am
Google Maps

Google Business Profile Owner/Manager Access Invites Not Working For Some

Jun 15, 2026 - 7:21 am
Google

Google Shopping Results With Good Price Label

Jun 15, 2026 - 7:11 am
 
Previous Story: Wikipedia Search Engine Wikiseek