The dependence of search result quality on user-generated data: evidence from an experiment

Abstract

The quality of search results differs across search engines. Do these differences arise because of differences in the search algorithm or because of different data sets about past usage behavior? We report evidence from an experiment with a small search engine. Our main finding is that the small search engine can produce equally good search results than the largest search engine (Google) for popular queries, but not for infrequent tail queries. This suggests that differences in search engine quality are largely driven by the amount of data a search engine has collected from its users in the past. Because the vast majority of queries are tail queries, this result has high policy relevance, shaping the governance of data-driven industries (e.g. in the EU’s Digital Markets Act).