Person:

Gehlbach, Hunter

Increasingly colleges and universities use survey results to make decisions, inform research, and shape public opinion. Given the large number of surveys distributed on campuses, can researchers reasonably expect that busy respondents will diligently answer each and every question? Less serious respondents may 'satisfice,' i.e., take short-cuts to conserve effort, in a number of ways-choosing the same response every time, skipping items, rushing through the instrument, or quitting early. In this paper we apply this satisficing framework to demonstrate analytic options for assessing respondents' conscientiousness in giving high fidelity survey answers. Specifically, we operationalize satisficing as a series of measurable behaviors and compute a satisficing index for each survey respondent. Using data from two surveys administered in university contexts, we find that the majority of respondents engaged in satisficing behaviors, that single-item results can be significantly impacted by satisficing, and that scale reliabilities and correlations can be altered by satisficing behaviors. We conclude with a discussion of the importance of identifying satisficers in routine survey analysis in order to verify data quality prior to using results for decision-making, research, or public dissemination of findings

When ordering items on attitude/opinion questionnaires, do survey designers bias respondents’ answers by the mere act of choosing to organize their survey in a particular way? We hypothesize that, under specific frequently-occurring conditions, respondents employ an anchoring and adjusting strategy in which their response to an initial survey item provides a cognitive anchor from which they (insufficiently) adjust in answering the subsequent item. Three experiments indicate that respondents anchor and insufficiently adjust in certain situations, anchoring and adjusting leads to higher inter-item correlations between adjacent items, and these inflated correlations can (spuriously) increase the reliability estimate of the scale that they comprise and affect the resultant correlations with other measures. These effects are not consistently accounted for by a “superior memory search” explanation. In organizing their surveys, researchers may wish to combat this bias by intermixing items designed for different, but related constructs.