A Declarative Specification for Machine Learning Architectures

Abstract

Going from the description of a model architecture in a figure to its implementation can be a fraught process. This work presents a markup language for specifying model architectures; an associated Python package used to convert models into a runnable format; utilities to import the model into PyTorch; a publicly available model repos- itory; and a tool to visualize the resulting models. This paper also provides a review of some popular machine learning architectures, examples of models created using the markup, and associated experiments. The system is named Agrippa * , determined through a bracket-poll tournament conducted by the author’s roommates. The language simplifies certain aspects of model development: parameters are named, explicit, and can be specified as being frozen or shared; models can be imported into different projects with few code changes; coherent parts of the architecture can be or- ganized into self-contained blocks; and parameter initialization techniques are explicit. The language syntax is simply XML. Models compiled using the Agrippa Python pack- age are converted into the ONNX format, a neural network interchange format sup- ported by a variety of machine learning frameworks. The web component of Agrippa can be found at http://agrippa.build , which con- tains the visualization tool, model repository, and documentation. † At this stage of development, the system has a number of limitations: models may not be compiled if they are larger than 2GB due to a formal limit imposed by the ONNX file format; not all operations available in ONNX are supported by the com- piler; the visualization tool does not yet support visual editing; and certain techniques, like dropout layers and batch normalization, require workarounds. There are a number of plausible ways each of these limitations may be addressed in the future.