About six years ago, Google published a behind the scenes video of what goes on in an internal search quality meeting. Now, CNBC published an article after being invited to sit in on one of these meetings this past month. It seems to me, outside of some big names, the meeting structure and format has not changed much in the past six years.
Here is what was described by CNBC about this recent meeting:
The team presented their data to Gomes and Google fellow Pandu Nayak, who leads ranking. The whole process, from the introduction of the idea to the conclusion of the meeting, took roughly 20 minutes.The team ran through various data points, like what percent of users clicked through a picture-link and then quickly clicked back (a bad sign), or whether there was a significant increase in the time until they made their first interaction with the results (also bad).
They showed some examples of queries where the pictures weren't helpful. The results for "Pomona College," for example, provided generic pictures of students.
However, the data ultimately showed that 91 percent of the time, raters found that image results were useful. In the live experiment, real users clicked through the pictures, too. Weighing that positive feedback against a slight increase in latency (how long it took the results page to load), Gomes and Nayak approved the tweak.
"For any change, the question is always, on balance, is it more useful than not?" Gomes said. In this case, it was.
The meeting wasn't exciting. There were no passionate debates or philosophical explorations of whether Google should be showing users more images. The data drove the decision.
That's by design.
"We've got a rigorous process of testing things out," Gomes said. "And we're really driven by metrics — that's the core of how we operate."
If you watch the meeting video from six years ago, it feels like it has been run the same way since then:
Of course, people like Amit Singhal and Matt Cutts are no longer at Google. But Panda Nayak, who was on Amit's right side is now leading up these meetings. I believe Paul Haahr is still at the meetings and so is Ben Gomes.
But data is what leads the meetings for the past several years and data will probably lead these meetings going forward for the next several years.
It is just interesting to see from that article that it SEEMS that not much has changed with these meetings, at least on how it is run, for the past several years. Of course, It would be super interesting to see another video capture a more recent meeting.
Forum discussion at WebmasterWorld.
Note: This story was pre-written and scheduled to be posted now, I am currently offline for a holiday.
