Interpretable selection and visualization of features and interactions using Bayesian forests

Abstract

In analyses of scientific data, it is often of interest to learn which features and feature interactions are relevant to the prediction task. We present here Selective Bayesian Forest Classifier, which strikes a balance between predictive power and interpretability by simultaneously performing classification, feature selection, feature interaction detection and visualization. It builds parsimonious yet flexible models using tree-structured Bayesian networks, and samples an ensemble of such models using Markov chain Monte Carlo. We build in its feature selection capability by dividing the trees into two groups according to their relevance to the outcome of interest. Our method performed competitively compared to top classification algorithms on both simulated data sets and real data sets in terms of classification accuracy, and often outperformed these methods in terms of feature selections and interaction visualizations.