How Sensitive Is Your Data: Sensitivity Analysis for Missing Data and Enhanced Tipping Point Displays for a Simulated 2^2 Factorial Designed Experiment

Abstract

Missing data is a prevailing issue for statisticians and medical practitioners, who must deal with incomplete information in data from many sources, including clinical trials and other randomized experiments. This issue is especially worrisome in critical cases, such as policy making or medical drug and device approval, because it is difficult to determine how missing units might affect conclusions about efficacy. Numerous studies have discussed methods for handling missing data, but there is no universal agreement on how to interpret results. One promising solution comes from tipping point analyses. Liublinska et al. (2014) visualized results from a sensitivity analysis via enhanced tipping point (ETP) displays which aided the process of an FDA drug evaluation. While the use of ETP displays has not been formalized for 2^k factorial experiments, increasingly advanced computational tools have made such a generalization feasible. I therefore propose extending the use of ETP displays to balanced 2^k factorial experiments. Based on the mathematical properties of estimators for factorial experiments, I show how this process works specifically for balanced factorial designs. Then, using a simulated data set, I run a study of a factorial experiment where k=2 and compare the results for cases of high and low "missingness.” From my analysis of a 2^2 design, I then generalize my procedure to 2^k designs. This is a powerful and useful tool for factorial experiments as they, too, are subject to the problems of missing data. It is now clear that these convenient displays can also be used for 2^k experiments.