The Process of Improvement to Google’s Search Algorithms
A detailed explainer published by Google describes how changes to its search engine are evaluated internally before making it available to its users.
Danny Sullivan, Google’s Public Liaison for Search, explains the role of Google’s research team, search quality raters, and in-product experiments to enhance its features through a comprehensive process of evaluation.
Whenever a user searches for some information online, sorting is done by search engines through billions of web pages to look for the most relevant and helpful resource.
As online content and web keep on changing, search engines need improvements to cater to those changes. Google makes use of the subsequent ways to update its search algorithms based on quality feedback.
The research team at Google focuses on getting feedback from people around the world to make the Search more useful. It is an understood fact that people from different geographical regions have different information needs, their interests and languages are not the same.
According to Google “we invite people to give us feedback on different iterations of our projects and we do field research to understand how people in different communities access information online”.
One such example is Google Go which works best on phones that are less powerful and regions that do not have a reliable internet facility. This app was developed after getting valuable feedback from people from emerging markets.
Search Quality Raters
“Search is designed to return relevant results from the most reliable sources available”. This phrase is the essence of 160-page rater guidelines that describe in great detail how Google’s systems intend to surface great content.
Insights about relevance and reliability are provided by a group of more than 10,000 Search Quality Raters which is used to improve search algorithms.
A sample of queries is assigned to a bunch of raters and two sets of pages are shown to them; one is the current version of Google and the other with the improvements. Raters evaluate each page against the query and assess whether the page meets the information needs based on their judgement. To gauge things like expertise, authoritativeness, and trustworthiness—sometimes referred to as “E-A-T”—raters are asked to try and do reputational research on the sources.
It is pertinent to mention here that these evaluations or ratings do not determine the rankings of a particular web page in Search.
The third way to analyze changes to Google Search is through in-product experiments. These are live experiments that are designed to get insights on how people interact with a new feature. The impact of improvement is measured by providing new features to a small percentage of users.
According to Sullivan, questions like “Did people click or tap on the new feature? Did most people just scroll past it? Did it make the page load slower? can help us understand quite a bit about whether a new feature or change is helpful and if people will actually use it”.
In 2019, we ran more than 17,000 live traffic experiments to test out new features and improvements to Search. If you compare that with how many launches actually happened (around 3600, remember?), you can see that only the best and most useful improvements make it into Search.
The evaluation process is an effective and on-going exercise that enables Google to make frequent improvements according to the needs of people based on demographics.