Endogenous Sample Selection

Abstract

This paper examines enumerator incentives in the production of survey data, a crucial input to research and policymaking. In theory, survey data is generated from a randomly selected, representative sample of the population. We provide causal evidence that in practice, enumerators respond to variation in effort cost across survey subjects by excluding high-cost subjects. To this end, we exploit the random assignment of individual questionnaires across 3.4 million households in 181 Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), two prominent global household survey programs, as a source of exogenous variation in effort cost. We find that in 110 (39) of these surveys at least 5% (10%) of survey subjects eligible for individual questionnaires are missing from the sample. Missing individuals differ systematically from included individuals. As a result, survey samples are not representative of the population, causing bias in key aggregate statistics. In particular, we show that the fertility transition in Africa – where few alternative data sources on fertility exist – is likely to have been significantly faster than reported, leading to lower fertility levels today. Finally, we provide suggestive evidence that endogenous sample selection affects a wide range of surveys, thereby highlighting the broader relevance of the novel quantity-quality trade-off in data collection documented in this paper.