Randfish, popular amongst the forums, has posted a thread at Search Engine Watch Forums named OpenRank - An Open Source WWW Index. Basically, he started OpenRank "with the goal of creating an open source index of web pages." In the thread there is a lot of confusion about the intent of the project. Randfish clarifies that this will not be used for searching, which seemed clear from the introduction, but it will be used to gather data for Webmasters to better understand where they stand in terms of the whole Web in search engines.
1. Create an alternative to PageRank using the link structure stored in the database. This new measure of global popularity would be both "accurate and precise" and could help people see exactly what links are influencing their "OpenRank"2. Create a measurement system of local popularity by segmenting pages into subject-specific communities and then examining the popularity of pages based on their links within those communities.
3. Create tools that would conduct more advanced and comprehensive link analysis then is currently available using the APIs or link commands at the search engines. One could, for example, see the exact number of links to each web page with a particular anchor text or view all the links sorted by anchor text, or by extracted page topic or poularity, etc.
Besides for this being a huge project to undertake both technically and financially, I am not sure how practical it is. Most search engines indexes, i believe, do not overlap that much. There were studies done that Yahoo! index has maybe 30% (not sure on the figure) overlap with Googles. If this is the case, then one can imagine the difficultly with this. Spidering the Web is a huge task, we built some small spiders and its not easy. I respect and wish this project the best, I will do what I can to support it.
Visit OpenRank at http://www.openrank.org/.
