Peeking Into Google Reveals More of Google's Architecture

Mar 13, 2006 - 7:55 am 2 by

An article at InternetNews.com from March 2nd, has Google vice president of operations and vice president of engineering, Urs Hoelzle revealing some of the "behind-the-scenes tour of Google's architecture."

Bill Slawski at Cre8asite Forums created a thread on this article named Google's architecture, Informative news story where he pulled out a couple quotes.

Google replicates the Web pages it caches by splitting them up into pieces it calls "shards." The shards are small enough that several can fit on one machine. And they're replicated on several machines, so that if one breaks, another can serve up the information. The master index is also split up among several servers, and that set also is replicated several times. The engineers call these "chunk servers."

The company also is applying machine learning to its system to give better results. Theoretically, he said, if someone searches for "Bay Area cooking class," the system should know that "Berkeley courses: vegetarian cuisine" is a good match even though it contains none of the query words.

To do this, the system tries to cluster concepts into "reasonably coherent" subclusters that seem related. These clusters, some tiny and some huge, are named automatically. Then, when a query comes in, the system produces a probability score for the various clusters. This kind of machine learning has had little success in academic trials, Hoelzle said, because they didn't have enough data. "If you have enough data, you get reasonably good answers out of it."

The article is definitely worth a read and then join the forum discussion at Cre8asite Forums.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: August 26, 2025

Aug 26, 2025 - 10:00 am
Google

Google: We're Testing Changes To AI Mode To Encourage Clicks

Aug 26, 2025 - 7:51 am
Google Search Engine Optimization

Google Search Console Link Report Doesn't Have A Last Updated Date

Aug 26, 2025 - 7:41 am
Bing Ads

Microsoft Responds To Hard To See Search Ad Labels

Aug 26, 2025 - 7:31 am
Other Search Engines

Sparktoro Research: Traditional Search Is Not Declining

Aug 26, 2025 - 7:21 am
Bing Ads

Dos and Don'ts Of Microsoft Advertising Suspensions & Appeals

Aug 26, 2025 - 7:11 am
 
Previous Story: Possible Yahoo! Search Update