Entries from Search Engine Roundtable tagged with 'slurp'

It seems like some webmasters are becoming fed up with the activity of Yahoo's crawl, Yahoo Slurp, relative to the amount of traffic Yahoo Search is sending the web site. In fact, some webmasters have taken the plunge and banned...

Posted on June 30, 2008 7:35 AM

Last week we reported on a Yahoo update and a new method of crawling. The new crawl behavior is supposed to help the Yahoo bot, Slurp, be more efficient on your site. It seems that many SEOs and Webmasters are...

Posted on August 27, 2007 8:05 AM

WebmasterWorld and DigitalPoint Forums thread report a spike in activity with Yahoo Search's web crawler, Slurp. People have noticed a large increase in page hits and bandwidth usage, caused by Yahoo! Slurp recently. I run several sites and I have...

Posted on July 2, 2007 8:20 AM

The Yahoo! Search Blog has announced that webmasters will now see Yahoo's spider, Yahoo! Slurp, returning a new domain name in your logs. The same IP addresses now render to the domain name crawl.yahoo.net and no longer return the domain...

Posted on June 8, 2007 11:38 AM

A WebmasterWorld and Search Engine Watch Forums threads are both reporting issues with Yahoo! Slurp (Yahoo!'s Crawler) indexing pages they should not be, and in quantities that may be harmful. It appears that only specific bots are not obeying the...

Posted on January 17, 2007 7:57 AM

Thursday night, last week, the Yahoo! Search Blog wrote Yahoo! Search Crawler (Yahoo! Slurp) - Supporting wildcards in robots.txt. I am honestly a bit shocked by the SEO community's response to this, or lack thereof. I have spotted two threads...

Posted on November 6, 2006 7:14 AM