Microsoft ContentAds Publisher Program Pilot Sign Up Form | Main | Yahoo! Search Marketing Announces New Customer Support Hours

Yahoo Slurp, Yahoo Search Crawler, Suffering From ADHD?

A DigitalPoint Forums thread has dozens of reports that Yahoo Search's crawler, Yahoo Slurp, took some bad medicine recently. Many are reporting that they see the crawler spidering their sites like never before. Some times they have seen the spider multiple to over a 1,000 crawls at one time.

The first report claimed about 500 Yahoo spiders:

I have a forum and I get 500+ (sometimes 800+) Yahoo spiders daily. Why is there so many? I only get 1 or 2 Google spiders.

Others reported similar cases with much more Yahoo spider activity then Google spider activity. These reports seemed to have died down recently.

The initial report came at Friday afternoon and then died out Saturday afternoon. So maybe it was just a temporary Yahoo glitch?

Forum discussion at DigitalPoint Forums.

Update: Yahoo sent me a statement update this behavior:

On March 29th and 30th, some of you have noticed Yahoo! Slurp spidering your website more than usual – this temporary blip was inadvertently caused by a major crawl infrastructure upgrade we have been doing for last 1 month.

While the Yahoo! Slurp should be stable now, if you continue to notice unreasonable traffic from Yahoo! spiders, please provide feedback at the Site Explorer Suggestion Board.

Thanks,

Yahoo! Search Team

posted rustybrick in Yahoo! Search Optimization at March 31, 2008 7:48 AM Comments (2)

Comments

This is nothing new. This has been true for as long as I can remember.

 

Not at this level...

 

Post a comment

Do you want us to save your personal Information?


To subscribe to the Search Engine Roundtable, click here