Question-Driven Reasoning in AI-Assisted Decision-Making: A Content-Based Approach

Abstract

We introduce the Question-Driven Theory of AI-Assisted Decision-Making, a content-based model of reasoning to explain human-algorithm interaction in high-stakes risk assessment contexts. According to the content-based theory of reasoning proposed by Koralus et al., human reason is driven by the goal of reducing the complexity of our questions as directly as possible, as opposed to alternative reasoning models that focus on maximizing expected utility by considering all possible alternatives. By emphasizing the role of the question-answering mechanism in human reasoning, this theory allows us to bring a new perspective to theoretically model and thereby understand the dynamics of human-algorithm interaction. According to our theory, AI-assisted decision-making can be understood in two phrases, the Question-Raising phrase and the Question-Answering phase. First, the algorithm prediction guides the decision-maker to raise an actionable question, while adding dependencies of concepts into their mental model representation. Then, the decision-maker reaches a decision by settling the question with their mental model representation. We propose the design of and conduct a behavioral experiment to test the applicability of a content-based theory on AI-assisted decision-making. We show that the presentation of an algorithm’s risk assessment predictions, such as logically equivalent information with either positive or negative probabilities, can influence the type of questions decision-makers pose and, subsequently, the decisions they make. We propose a framework to computationally model this procedure, as well as normative directions for designing AI assistance that prompt the right kind questions to ensure rational decision-making procedure and fair results.