RustySearch Hits 5,000 Rated Searches

May 12, 2005 • 3:38 pm | comments (9) by twitter Google+ | Filed Under Search Engine & SEO Theory

I would like to thank you for Helping to Promote RustySearch and the Search Engine Relevancy Challenge. We have surpassed 10,000 searches of which 5,000 searches were rated. Please continue to promote it, so we can hit 20,000 rated searches. The graphs and charts below are current as of 5/12/2005 at 3:30PM (EST) pulled from The Search Engine Relevancy Challenge, RustySearch search engine. I'll explain each graph and chart below.

The "Search Engine Relevancy Dials" below show how each search engine scored on average in terms of relevancy on a one through five scale at this point in time. The search engine with the highest score is considered the most relevant search engine voted by you.

Search Engine Relevancy Dials



Ask Jeeves

MSN Search


This link graph groups search engines by rating. The reason we plotted it on a graph like this is to show you that there is this U shaped curve that is consistent between all search engines when rated. In our opinion, it means that most people either feel the results are relevant or not relevant. Very few people feel that a search engine can be "somewhat" relevant.


Finally, here is a raw summary count of data that we placed on a simple chart view for you. This data is real time and will continue to update as people rate. The averages from top-down are the average rating count by search engine. The averages from left-right are the average rating count by rating group. The value at the far bottom-right corner is the total rated search results obtain at this point in time at RustySearch.

Raw Summary Data
Search Engine12345Average
Ask Jeeves369121112110537249.80
MSN Search422111105120470245.60

Previous story: IP Detection for Currency Content Delivery



05/14/2005 01:24 am

Hello, I'd loooooooooooooove to see how Gigablast stacks against these 4 search engines: Wisenut, too. Without including those players in your comparison, you are not giving them a fair chance.

PJ Brunet

05/14/2005 04:21 am

Hasn't Altavista been around longer than MSN?

Ben Parzybok

05/14/2005 07:01 am

Hi, This is an excellent idea and I'm excited to continue to track the results. By the way, I think your search form is choking a bit on apostrophes. For example, try a search for: What's the capital of Washington? The form results to: What the search seems to work, but after you rate it and return back to the results, it then re-searches and comes back with results for just "What" Another thing I noticed about searching for What's the capital of Washington is that some search engines answer the question right at the top, making, in my opinion, the results very relevant. However RustySearch doesn't take those auto-answer results into consideration which I think are going to skew your statistics a little. Only Askjeeves hits it at #1 with an actual result but Google answers the question in the number 1 space, but goes willy nilly with the links (to D.C.) Same deal with msn Olympia is nowhere to be found in Yahoo's top 10.

Dmitry Yeskin

05/14/2005 11:19 am

hi! Just found your rustysearch engine. Trying to determine how it's work. But in overal I like an idea to rate a results. Good luck!

Barry Schwartz

05/15/2005 01:21 am

Otis, I will be conducting more tests in the future for the "smaller" search engines. But this is the first of many. So thanks for your input. PJ, Alta Vista is powered by Yahoo!. So by rating Yahoo!, you are rating AltaVista as well. Ben, thanks for the technical feedback, we will look into that asap. Regarding the "SmartAnswers", you're right, and it is hard to include without it tipping off the user as to which engine is providing the answers. Dmity, thanks!

Eli Feldblum

05/16/2005 02:36 pm

Nice job, Barry. No surprise there for most of us, I guess. I've always felt that Yahoo! delivered the most relevant results. What I find interesting is the study present at SES San Jose last August that most users find Yahoo! most relevant WHILE using it, but if asked later, they report that Google worked best.

Chris Nielsen

05/19/2005 09:21 pm

Well, it seems that the results show what I expected, that they are all about the same with the exception of the newcomer, MSN. Or, that people's perception of them are all about the same... :-)

Kim Stian Ervik @ SEO Norway

05/20/2005 03:27 am

The relevancy for Yahoo might be best for USA related search, but they are no competitor for Google on European searches and aspecially Norwegian searches.

Natasha Robinson

12/29/2005 03:34 am

OK! So if your like me... then you spend a lot of time in the SERPs. And as of late I find myself getting better results from MSN (even though I still check their results 3rd, force of habit I guess). So I was wondering if you'd be doing this test again. I'd love to see how the results have changed over time. Natasha "That Girl From Marketing" Robinson

blog comments powered by Disqus