Yahoo! Slurp Now Located at crawl.yahoo.net

Jun 8, 2007 - 11:38 am 0 by

The Yahoo! Search Blog has announced that webmasters will now see Yahoo's spider, Yahoo! Slurp, returning a new domain name in your logs. The same IP addresses now render to the domain name crawl.yahoo.net and no longer return the domain name inktomisearch.com.

As of today, the transition is complete and all machines crawling as Slurp are now in crawl.yahoo.net. You can see this change in your web server logs, where the page accesses from inktomisearch.com are being fully replaced by crawl.yahoo.net contacts. Note that this does not cover other Yahoo! crawlers, such Yahoo! China, and other verticals, like Yahoo! Shopping, Yahoo! Travel, etc., which have their own user-agent.

The good news: you don't have to change your robots.txt file over to prevent Yahoo! from crawling your site (if that's what you're aiming to do) because Yahoo says that the crawler user-agent is still Yahoo! Slurp and the IP addresses have not changed.

Discussion continues at WebmasterWorld.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Flux, AdSense Ad Intent, California Link Tax & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Updates

Google March Core Update Still Rolling Out & Heated SEO Chatter Continue

Apr 25, 2024 - 7:51 am
Google

Report: How Prabhakar Raghavan Killed Google Search

Apr 25, 2024 - 7:41 am
Google Search Engine Optimization

Google Favicon Documentation Adds Rel Attribute Value Definitions

Apr 25, 2024 - 7:31 am
Google Ads

Google Ads API Version 16.1 Now Available

Apr 25, 2024 - 7:21 am
Google Search Engine Optimization

Google: Splitting & Merging Sites Takes Longer Than Normal Site Migrations

Apr 25, 2024 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: April 24, 2024

Apr 24, 2024 - 4:00 pm
Previous Story: Google Maps: Invading Your Privacy? (Not Anymore!)