Paul Haahr, who has been with Google for over 16 years, working on core search problems and solutions posted a 20 tweet thread on why Google ranks what they rank. It stems from an SEO challenging the morals behind how Google's ranking algorithm works.
I feel it is important to share the tweets with everyone, because they are thoughtful and do show a lot of the reasoning behind why Google search does what they do.
I read your essay a couple of times. Thanks for putting the thought into it, though I have to admit that the over-the-top language undercuts your message. When an essay describes me or my employer a “pimp” and a “dictator,” it’s hard to take seriously. /1
— Paul Haahr (@haahr) June 11, 2018
Still, I’d like to try to give some clarity on my thinking here, because you’re obviously very passionate about search. Again, me personally, I can’t speak for Google as a whole or the diversity of opinions among its employees. /2
— Paul Haahr (@haahr) June 11, 2018
As I said in our Twitter exchange, Google Search doesn’t claim to know “truth” or to have special insights into “The Good.” But we do have a mission statement, “Organize the world’s information and make it universally accessible and useful.” I think that’s a pretty good goal. /3
— Paul Haahr (@haahr) June 11, 2018
When you question whether Google “should be the ones rating businesses in the first place,” I’m left asking how else would we organize the world’s information? For me, this is a practical matter, because that’s what we’re openly aspiring to do. /4
— Paul Haahr (@haahr) June 11, 2018
The task for us is to rank web pages. It’s still at the core of search. I’m not going to talk about how it’s implemented (or confirm or contradict your guesses), but this is a thing we have large teams thinking about. /5
— Paul Haahr (@haahr) June 11, 2018
But, as a process, how do we do it? The answer we’ve evolved towards includes evaluation based on clear, public guidelines about what good results are. This grounds search in something other than our own opinions. /6
— Paul Haahr (@haahr) June 11, 2018
That the guidelines are public is, for me, a big deal. They represent a lot of work and we’d considered them proprietary for a long time. But, to provide transparency and enable criticism, they’re now open for anyone to read. (Hopefully, they’re also useful in IR research.) /7
— Paul Haahr (@haahr) June 11, 2018
Asking raters about “highest quality” pages is no euphemism. Understanding which pages they trust, which they think have good information, etc, is important. Not trying to get this stuff right would be doing a disservice to our users. /8
— Paul Haahr (@haahr) June 11, 2018
You’re right that Google search could just return alphabetical lists of pages. But it wouldn’t be useful. Or only Mom & Pops. Again, not useful. We run experiments all the time, but these don’t seem particularly interesting, because I don’t know what we could learn. /9
— Paul Haahr (@haahr) June 11, 2018
But when you write “Perhaps random set of worthy enough businesses as determined by rules setup by us, the public you serve?” I’m totally lost. The idea sounds worthy, but I have no idea how we’d get rules from the public. Especially, rules that work. /10
— Paul Haahr (@haahr) June 11, 2018
I say this because we look at lots of ranking ideas every day. We test them, running offline and live experiments. We analyze them heavily. A few work; most don’t. At the risk of sounding like a technocrat, it takes expertise and effort to move the needle here. /11
— Paul Haahr (@haahr) June 11, 2018
Similarly, when you write wanting a “chance for us to officially determine what we shall officially consider to be a “low quality” website,” even if there were a way to outsource that decision to users, doing so at the scale of the web seems intractable. /12
— Paul Haahr (@haahr) June 11, 2018
Like you, I think liberal democracy is the best way to organize society as a whole. I don’t think it’s inherently the right way to organize companies, product development, or research. I can think of few successful examples of, say, voting-based product design. /13
— Paul Haahr (@haahr) June 11, 2018
Where you get less philosophical and more concrete seems to be about things like manual spam actions, which are a necessary part of keeping search usable. A lot has been said about these and what Matt Cutts or John Mueller said in the past is better than anything I would say. /14
— Paul Haahr (@haahr) June 11, 2018
I disagree with the characterization of spam fighting as “censorship.” A search engine that doesn’t do it will be overrun and become useless, a tragedy of the commons. Protecting the spaces they offer is an obligation for successful services. /15
— Paul Haahr (@haahr) June 11, 2018
For what it’s worth, I’m sure everyone who works on search engines (and other web services) would love a world without spam and abuse. It makes a lot of things that should be easy, hard. “If men were angels…” /16
— Paul Haahr (@haahr) June 11, 2018
The point about offering public search stats is interesting. My sense is that most websites wouldn’t want Google to disclose how much search traffic they’re getting. There are also risks of such things becoming gamified. /17
— Paul Haahr (@haahr) June 11, 2018
Google engineers are not who you should speak to about regulation of Google. Have that conversation with policy makers, legislators, regulators, etc. That said, I try to act as if everything I do is being scrutinized by a reasonable regulator. I think all my colleagues do. /18
— Paul Haahr (@haahr) June 11, 2018
Finally, I wish you had engaged seriously in the question of how to improve the guidelines, because your passion for improving search could help us/you/the world. And I’m sure you would have many constructive criticisms if you moved down from the meta level. /19
— Paul Haahr (@haahr) June 11, 2018
I’m going to end here. To quote your essay, “the amount of time I can take away from productive hours to do philosophy” is limited. Please, keep up the criticism. /20
— Paul Haahr (@haahr) June 11, 2018
Hope you enjoy this read as much as I did.
Forum discussion at Twitter.
