Peeking Into Google Reveals More of Google's Architecture

Mar 13, 2006 - 7:55 am 2 by

An article at from March 2nd, has Google vice president of operations and vice president of engineering, Urs Hoelzle revealing some of the "behind-the-scenes tour of Google's architecture."

Bill Slawski at Cre8asite Forums created a thread on this article named Google's architecture, Informative news story where he pulled out a couple quotes.

Google replicates the Web pages it caches by splitting them up into pieces it calls "shards." The shards are small enough that several can fit on one machine. And they're replicated on several machines, so that if one breaks, another can serve up the information. The master index is also split up among several servers, and that set also is replicated several times. The engineers call these "chunk servers."

The company also is applying machine learning to its system to give better results. Theoretically, he said, if someone searches for "Bay Area cooking class," the system should know that "Berkeley courses: vegetarian cuisine" is a good match even though it contains none of the query words.

To do this, the system tries to cluster concepts into "reasonably coherent" subclusters that seem related. These clusters, some tiny and some huge, are named automatically. Then, when a query comes in, the system produces a probability score for the various clusters. This kind of machine learning has had little success in academic trials, Hoelzle said, because they didn't have enough data. "If you have enough data, you get reasonably good answers out of it."

The article is definitely worth a read and then join the forum discussion at Cre8asite Forums.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Weekend Volatility, Google On Search Leak, Elizabeth Tucker Interview & Apple Intelligence - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: June 17, 2024

Jun 17, 2024 - 10:00 am
Google Search Engine Optimization

The Maddening Adventure Of Tracking AI Overviews In Google Search Console

Jun 17, 2024 - 7:51 am
Google Ads

Google Ads Forcing Some Advertisers Off Credit Card Payments

Jun 17, 2024 - 7:41 am
Apple App Store

Apple Updates Applebot Docs With Applebot-Extended, Reverse DNS & More

Jun 17, 2024 - 7:31 am
Google Maps

Google Maps Updates Fake Engagement Policy With More Detail

Jun 17, 2024 - 7:21 am
Google Search Engine Optimization

Google Updates hreflang Documentation While I Was Offline

Jun 17, 2024 - 7:11 am
Previous Story: Possible Yahoo! Search Update