Moving to a U.S Hosted Server: Will it Impact Country Specific SEO? | Main | Use Google Ad Manager To Set Minimum AdSense eCPM

Some Webmasters Are Banning Yahoo Slurp

It seems like some webmasters are becoming fed up with the activity of Yahoo's crawl, Yahoo Slurp, relative to the amount of traffic Yahoo Search is sending the web site. In fact, some webmasters have taken the plunge and banned Yahoo from crawling their sites.

It is funny, because some webmasters were or are afraid to ban Yahoo because they think it might have some sort of impact on their Google results. One such webmaster got over it and decided to ban Yahoo despite his fears.

There is a thread at WebmasterWorld discussing the pros and cons. Some also wonder that will banning Yahoo have a negative impact on the Yahoo SearchScan & McCafee partnership and might have your site marked as harmful? I doubt it.

If you want to block Yahoo Slurp, learn how to at this page.

Forum discussion at WebmasterWorld.



Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!

posted rustybrick in Yahoo! Search Optimization at June 30, 2008 7:35 AM Comments (16)

Comments

Where I get the lions share of my traffic from Google, the conversion rates I get from Yahoo are so much better that I get from anywhere else. I will keep Slurp coming around for a while.

 

I agree with the previous commentor. Although the traffic Yahoo! sends is small in volume (about 8% or so for most of my sites) the conversion rate and the average order value this traffic delivers is generally better. I'm quite happy with others pulling out of Yahoo! makes my positions better :p

 

Those who block are ..

 

I have banned Slurp from my forums many times through the years. It tends to get caught up in the crawling process. But after a couple of weeks I'll let it back in.

Once or twice I have had to ban Googlebot, too. I think this is a problem endemic to large complex sites with a lot of tight interlinkage, such as you have on forums and blogs.

 

It could be that Yahoo is trying something new with Slurp and the frequency of visits has gone up for that reason.

 

It could be that Yahoo is trying something new with Slurp and the frequency of visits has gone up for that reason.

 

As for me I recieve better returns from Yahoo I believe for my business it is Demographics.

 

It's very interesting, because Live and Yahoo did not bring much users to my web sites... but they are there almost the same weight than Google... so... I'll take the idea of
User-agent: Slurp
Crawl-delay: 240

And use for both and see...

 

I don't see the problem. Traffic is traffic, it is a numbers game. The more traffic you get, the more sales you will make in the long run.

 

I really do hope Yahoo makes a come back. One search engine dominating us all is not a good thing for the market.

 

I agree with previous poster. Although I get about 4x more traffic from Google, my conversion numbers are around the same. Yahoo does have better targeted traffic.

 

We get 90% of our traffic from Google and only 5% from Yahoo...We will keep them around for now.

 

Apparently I have had hundreds of GB of traffic going to yahoo slurp on my site

I needed this to stop IMMEDIATELY....

I believe it is because slurp for some reason is downloading my FLV files - I have
no clue why it would need to do this but whatever.....

So I am testing some new code in my robots.txt file that hopefully should
eliminate this problem....


user-agent: *
Disallow: /flvplayer.swf

User-agent: Googlebot
Disallow: /*.flv$

User-Agent: Yahoo! Slurp
Disallow: /*.flv$


So if you have a website with various media files, you may be able to tell the bot
to not download them.

Here are the resources I used to build these directives.


http://www.google.com/support/webmasters/bin/answer.py?answer=40367


http://www.ysearchblog.com/archives/000372.html

 

I've often heard different Webmasters rant an rave about the virtues of different technologies.

One likes article marketing while another can't live without PPC.

In my opinion you need all of the above to succeed in Internet Marketing. Just because one technology may out perform another that is in and of itself no reason to ban that technology. Perhaps give more time to one activity as compared to another; but altogether banning has never made sense to me.

 

Lol, seems to me that if Yahoo traffic converts so well y'all might be better off optimizing for Yahoo and banning all those visitors from google, who don't make you any money.

It's not a "numbers game" it's a Highly targetd, converting traffic, game" and whoever sends the best converting traffic is the area I'm going to focus on.

Another thing you y'all might want to consider, regarding your google listings, is tightening them up so the traffic is better targeted.

Don't blame the SE for your traffic problems nor the traffic they send. The SE didn't optimize your pages, you did.

 

Does yahoo even listen to robot.txt??

 

Post a comment (Note: Can Take 120 Seconds For Your Comment To Show Up)

Do you want us to save your personal Information?


To subscribe to the Search Engine Roundtable, click here