Essays on Data-Driven Product Innovation in Organizations

Abstract

How do organizations anticipate market demand to produce commercially successful product innovations? Much previous research emphasizes the advantages of being "data-driven"—utilizing data analytics or experimentation to validate market demand for new products. This dissertation investigates when and why such methods can be beneficial, as well as the circumstances in which they may limit innovation within organizations. In three empirical essays, I employ various theoretical lenses, such as behavioral strategy, decision-making, and organizational theory, to develop new theories on how and why data-driven methods systematically affect innovators' perceptions about the likelihood of success for their innovations. The first essay employs a unique dataset comprising product-level sales data and employee résumé data from the consumer product sector, demonstrating that the efficacy of a data-driven approach in fostering innovation depends on the organization's methodological pluralism—specifically, the balance between quantitative and qualitative methods. The second essay elucidates one underlying reason why demand for new innovations can be challenging to predict using data-driven techniques. I create an agent-based model to demonstrate how diffusion dynamics obscure the observable demand for novel products, and I empirically validate the model using the same product-level sales data. The third essay presents an entirely different reason that data-driven decisions can mislead innovation—using data from developmental PC game development, I show that early enthusiastic users may provide demand signals unrepresentative of the broader market. A recurring theme throughout the dissertation is a unique perspective on how organizations achieve competitive advantage in innovation: by shaping the methods their members use to perceive opportunities.