The Importance of Random Effects in Variable Selection: A Case Study of Early Childhood Education

Abstract

Multilevel models are very common in statistical studies of education, as they account for the nested structure of many data sets. This paper combines this methodology with variable selection, motivated by the Early Learning Studies at Harvard (ELS@H). We use mixed-effects and fixed-effects LASSO in a simulation to gain insight into how random effects can impact variable selection. Furthermore, we also use the simulation to better understand the ELS@H data's weak relationship between the predictors and outcome.

From the simulation, we find that under settings of high cluster variance and high correlation between the clusters and covariate generation (which correspond to a low Intra-Class Correlation, or ICC), the fixed-effects and mixed-effects LASSO diverge in the variables they shrink. The ELS@H data shows a similar divergence, even though it exhibits a low ICC. We also find other general trends in how the fixed-effects and mixed-effects LASSO perform in various types of data. At the end of the paper, we are able to more strongly conclude that random effects are not contributing to the low predictive signal in the data. We also discuss whether considering random effects in variable selection is advantageous at large.