Tiger Here | Main | Quick Tiger Spotlight Review

The Search Engine Relevancy Challenge Live: RustySearch

A few weeks ago, I proposed the idea of The Search Engine Relevancy Challenge: Pepsi vs. Coke. In this challenge, I proposed building a "white labeled" search engine, that randomly selected results from one of the top four engines and asked you to rate the search engine results, individually, from one to five. Well, we have built the white label search engine.

Officially, this is being released tomorrow, but I wanted you guys to get to it first. There is more information at the official, The Search Engine Relevancy Challenge. I named the white labeled engine RustySearch and it works as follows:

1. Go to RustySearch (will open in new window)
2. Enter a search phrase and click search
3. Click on the title, the URL or the "rate this result" link. The site will open up in a framed window, so do not open the result in a new window.
4. At the top, there is a frame that says, "Rate the relevancy of this Web page for 'search phrase here'". Rate the result from one to five, one meaning a very poor relevancy and five meaning an excellent relevancy.
5. After you click a radio option, it should read, "Thanks for your input! Your vote (and many others) will decide who, really, is the search engine king!"
6. Then click on the button to take you back to the results
7. Continue on by rating the next result on that page, or search on a different result.
8. Repeat...

It is close to impossible to make everyone happy with a study on relevancy. Please try not to search on terms you are looking to rank well for. Search on terms when you have a information search need, outside of rank checking. This way, you can really test the relevancy as an end user and not an SEO.

I am off to Toronto SES, speak to you all tomorrow!



Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!

posted rustybrick in Search Theory at May 3, 2005 8:35 AM Comments (5)

Comments

Was thinking it might be easier to rate the page on a scale of 1-10 instead of having to vote 10 times on a page...could be as simple as how many of the top 10 are good pages?
maybe even 2 votes...how many are good, and how many are bad (rest would be "so so" pages)...just an idea.

 

Forcing the user to really see if the results and really spend time thinking if it is relevant.

You don't have to rate all 10, just rate the ones you want to rate.

 

The fact that results are numbered and in vertical alignment might tilt the results. There is a basic assumption (intended or not) that the higher up the more relevant a result is. I would recomend a grid layout, say three rows of three.

 

Hey Barry, the rustysearch logo looks a lot like the one at http://www.etrafficjams.com. Is that a stock graphic? Just curious. Nice viral campaign. Am looking forward to the results.
Lee

 

Stock images, so you know. Nothing gets more "White Label" then stock art.

 

Post a comment (Note: Can Take 120 Seconds For Your Comment To Show Up)

Do you want us to save your personal Information?

Premium Sponsors + advertise

To subscribe to the Search Engine Roundtable, click here