Improving Health Measures: Evidence From a List Experiment, Cognitive Interviews, and a Vignette Study

Abstract

Measuring health through surveys is challenging. Participants may respond in a socially favorable but untruthful way, and responses across respondents may be difficult to compare. List experiments and anchoring vignette techniques have been proposed to improve survey measures, and this dissertation applied qualitative and quantitative methods to evaluate either the usefulness or validity of those techniques. The first paper explored how list-based questions perform compared to direct questions when measuring two behaviors, smoking and intravenous infusion use. The difference-in-differences between the two survey methods and two behaviors was non-significant. List experiments might introduce downward biases rather than alleviate them due to cognitive difficulty in responding. The second paper applied cognitive interviewing to the design of vignettes among Chinese students with objectively different visual acuities. Ten major problems were identified regarding vignette comprehension, judgment, and responses. Respondents rated vignette character’s vision differently from their own, demonstrating a norm of being “strict with oneself and lenient towards others” in an Asian context. The third paper assessed the validity of three vignette survey methods (i.e., indirect comparison between self and vignettes, direct comparison between self and vignettes, and self-assessment primed by vignettes) in measuring distance vision. Surprisingly, it was found that vignette methods were not improvements over self-assessment, and indirect comparison performed the worst. These three studies shed light on a series of cognitive difficulties related to advanced survey techniques. Traditional self-assessment may be as useful as the list experiment and more valid than some anchoring vignette techniques in these studies’ context.