Thoughts on Index Size Debate

Aug 17, 2005 - 8:13 am 0 by

I have, for the most part, stayed out of the debate with the recent controversy over if Yahoo!'s index size is inflated or not. All I did was link to forum threads and articles on the topic.

I sat down with Gary Price, News Editor at Search Engine Watch, at the last SES conference and one of the many topics we discussed was this one. Forget the controversy over index size, I have no way of knowing if the 19.2 billion count is accurate or not. That is not the point...

To measure index size by conducting queries, in my humble opinion, is not a valid testing method. Why? Because search engines have an indexer component and a query component. Both of those are pretty much standalone. So for someone to do a query, no matter what it is, at one engine and compare the "total results" to an other search engine, to me, it seems very flawed.

Some of the reasoning on this can be found by a blog entry at The query processor can be limiting the results based on its matching and filter algorithms. One can not measure index size, when it goes through a query processor. In my opinion, it really doesn't matter that Yahoo! claims to be more then twice the size of Google and still doesn't show, for any of those queries, more total results. I know many people argue with me, but this is how I felt from day two - after speaking with Gary about the methodologies being used to measure Yahoo!'s claim.

Update: Jeremy Z just posted a few minutes ago his via on Of Course Size Matters! Got of love the blog-o-sphere.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Core Update Flux, AdSense Ad Intent, California Link Tax & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Maps

Google Business Profiles Gains Select Preferred Menu Source

Apr 23, 2024 - 7:21 am
Google Search Engine Optimization

Google: Crawl Budget Goes Across All Googlebot Crawling, Not Just Web Search

Apr 23, 2024 - 7:11 am
Blog Administration

Programming Note: Offline For Passover

Apr 22, 2024 - 4:05 pm
Search Forum Recap

Daily Search Forum Recap: April 22, 2024

Apr 22, 2024 - 4:00 pm
Google Search Engine Optimization

Google: We Need Very Few Links To Rank Pages; Links Are Less Important

Apr 22, 2024 - 7:51 am

Google Tests Discussion and Forums Questions With Answers Box

Apr 22, 2024 - 7:41 am
Previous Story: Inside AdSense Blog Launches