Differentiating Types of Meaningfulness as Motivation for Crowdsourcing Participation and Performance

Abstract

With the advent of powerful task performance platforms like Amazon’s Mechanical Turk (AMT), crowdsourcing has become a powerful means to address a variety of high-volume pragmatic problems, ranging from language translations to imagery analysis. Crowdsourcing uses crowds of relatively low-cost workers to perform tasks, finding accuracy in the volume of work done rather than relying on experts to address each requirement. This study investigated the breadth of meaningfulness as a potential incentive mechanism in driving performance, participation, and reservation wages in paid crowdsourcing using Amazon’s Mechanical Turk (AMT). Where previous research has equated meaningfulness to an altruistic and/or charitable purpose, this research investigated meaningfulness purpose beyond charity based on the insight that this incentive category, in previous work in industry and education, was intended to be more grounded in an individual objective of usefulness than in altruistic objectives. In this study, it was hypothesized that participants receiving charitable and profitable contexts to their tasks (e.g., informed that task outcomes were being used in a meaningful way) would produce higher performance and participation, as well as a smaller (or lack of) performance or participation drop-offs with lower pecuniary interventions (e.g., indicating a lower reservation wage). It was also hypothesized that participants receiving charitable contexts would produce higher performance and participation than profitable contexts, as well as lower reservation wages. Participants were recruited online using Amazon’s Mechanical Turk (AMT). Participants were asked to complete a demographics questionnaire, then complete up to ten trials of imagery tagging tasks focused on classifying compound emotional expressions, and finally were then asked to complete a subjective feedback questionnaire when they were finished. The results showed that participants in the charitable context case performed better than those in the no context control, but participants in the profitable context case did not perform better. There were no observed interaction effects on performance or participation between meaningfulness and pecuniary interventions. Participants in the mid-tier pecuniary intervention group ($0.10/trial) showed higher participation rates than those in the low ($0.05/trial) and high ($0.20/trial) groups, indicating a potential crowding out effect from the highest pecuniary intervention. This study provides initial evidence against broader interpretations of meaningfulness, pointing to a focus on using altruism and charity to incentivize crowdsourcing performance. However, there is room for broader investigation of meaningfulness characteristics to determine if other contexts might incentivize different participation, performance, or reservation wage outcomes.