Communicating Common Goal Knowledge Improves Trust-Calibration in Human-AI Collaboration

Abstract

In Human-AI collaboration, human agents often have a clear goal in mind and the AI Assistant tries to help users achieve their goals more efficiently. However, inferring users’ goals is non-trivial based on noisy user behavior and there is often mismatch in agent’s belief about each other’s knowledge of the ground-truth goal, leading to coordination failure. In this study, we propose that building common goal knowledge through communication improves human user’s mental model of the AI Assistant and leads to more efficient and effective human-AI collaboration. To test this hypothesis, we design an experiment where an AI assistant helps a human user shop for recipes on a grocery platform. We compare the user behavior and team performance under three experimental conditions: AI providing no information over its knowledge over human goals, AI expressing its belief over human’s goal through verbal communication(“Show”) and AI indicating its confidence in its belief over human’s goal (“Tell”). We find that communicating goal knowledge (in “Show” and “Tell”) increases user’s tendency to use AI when AI is indeed correct and improves user’s subjective ratings of the AI assistant.