The Effect of Priming Activities on Algorithm-in-the-Loop Decision Making

Abstract

Algorithms are increasingly being used in order to make predictions and decisions in a variety of contexts. However, it is often the case that humans use algorithms to inform their final decision. With this in mind, it is not only important that an algorithm is precise on its own, but it is also becoming important to understand how humans then use the algorithm. For example, in the criminal justice system, the algorithms used for risk assessments have been shown to discriminate against minorities. However, humans have also exhibited slight biases when using these algorithms. Thinking about the human-algorithm system as a whole could suggest a modification in the algorithm to account for human bias. While past studies have looked at how the presentation of the algorithm's prediction affects predictive performance and how a human incorporates it into his decision-making, this study looks at how a priming activity \textit{before} predictions are made affects predictive performance and how a human incorporates algorithms into his decision-making. This study asked Amazon Mechanical Turk workers to make predictions in two contexts: financial loans and the likelihood of defaulting, and pretrial release and the likelihood a defendant will either not show up to trial or commit a crime before trial. Some of these participants completed a priming activity related to the context beforehand, and some of these concepts were not provided with an algorithm's prediction. The results showed that simply completing a priming activity did not improve predictive performance. However, it did suggest that those more comfortable with a given context, as evidenced by the priming activity, had higher predictive performance. In addition, those that completed a priming activity generally weighted the risk assessment more than those that did not. Previous studies show a mix of these results. Some studies show that experts discount advice too much, often from overconfidence. This results in poorer predictive performance. Other studies show that those that are more comfortable with a context do discount advice more than those that are less comfortable but can still demonstrate higher predictive performance. This could suggest that priming activities should be designed to help participants become more acquainted with the context but not to the point of expertise or overconfidence.