Self-Structuring Machine Learning Models

Abstract

Machine learning models often use a large number of parameters in order to effectively approximate a function of interest on the entirety of its domain, leading to overfitting and high computational requirements. Towards the goal of reducing the number of parameters needed for good approximation, we introduce a new inference method in which a group of machine learning models self-assemble into a chain to collectively produce a single output for a given input. Each model in the chain takes input from the preceding model, produces an output, and adds another model to the chain to receive this output. The reuse of common units in each inference chain presents an opportunity for parameter count savings. In addition, we introduce an efficiency-aware metric that quantifies the predictive power of a group of models.