Can Non-full-probability Internet Surveys Yield Useful Data? A Comparison with Full-probability Face-to-face Surveys in the Domain of Race and Social Inequality Attitudes

Abstract

The authors investigate the potential utility of Web-based surveys of non-full-probabilistically sampled respondents for social science research. Specifically, they compare demographic, attitude response, and multivariate model results produced by two distinct survey modalities: the traditional full-probability sample face-to-face survey and the non-full-probability Web survey. Using data from the 2009 Race Cues, Attitudes, and Punitiveness Survey (RCAPS), the 2008 General Social Survey (GSS), and the 2008 American National Election Study (ANES), the authors find that (1) the unweighted demographic differences between surveys tend to be slight; (2) in comparison with GSS and ANES respondents, RCAPS respondents are more interested in politics and ideologically polarized; (3) in comparison with ANES respondents, RCAPS respondents are more racially and socially conservative, often selecting the most extreme response option; (4) when the dependent variable is a more general and abstract measure of social attitudes, the multivariate models generated by the ANES and RCAPS show several differences that are trivial in magnitude, but when the dependent variable is a more specific and concrete measure, the models show remarkable similarity; and (5) RCAPS multivariate models consistently explain more variance than ANES models. Overall, these findings show both substantial similarities across the two survey modalities as well as a few clear, reasonably well-specified differences.