Show simple item record

dc.contributor.authorSuzgun, Mirac
dc.contributor.authorGehrmann, Sebastian
dc.contributor.authorBelinkov, Yonatan
dc.contributor.authorShieber, Stuart
dc.date.accessioned2019-07-03T14:25:41Z
dc.date.issued2019-06-09
dc.identifier.citationSuzgun, Mirac, Sebastian Gehrmann, Yonatan Belinkov, and Stuart M. Shieber. 2019. LSTM Networks Can Perform Dynamic Counting. Proceedings of the ACL 2019 Workshop on Deep Learning and Formal Languages, Florence, Italy, August 2, 2019.en_US
dc.identifier.urihttp://nrs.harvard.edu/urn-3:HUL.InstRepos:40827356*
dc.descriptionMore information for citation: organization: Association for Computational Linguistics address: Florence, Italyen_US
dc.description.abstractIn this paper, we systematically assess the ability of standard recurrent networks to perform dynamic counting and to encode hierarchical representations. All the neural models in our experiments are designed to be small-sized networks both to prevent them from memorizing the training sets and to visualize and interpret their behaviour at test time. Our results demonstrate that the Long Short-Term Memory (LSTM) networks can learn to recognize the well-balanced parenthesis language (Dyck-1) and the shuffles of multiple Dyck-1 languages, each defined over different parenthesis-pairs, by emulating simple real-time k-counter machines. To the best of our knowledge, this work is the first study to introduce the shuffle languages to analyze the computational power of neural networks. We also show that a single-layer LSTM with only one hidden unit is practically sufficient for recognizing the Dyck-1 language. However, none of our recurrent networks was able to yield a good performance on the Dyck-2 language learning task, which requires a model to have a stack-like mechanism for recognition.en_US
dc.language.isoen_USen_US
dc.relationProceedings of the 2019 Workshop on Deep Learning and Formal Languages: Building Bridgesen_US
dc.relation.hasversionhttp://arxiv.org/abs/1906.03648en_US
dash.licenseLAA
dc.titleLSTM Networks Can Perform Dynamic Countingen_US
dc.typeConference proceedingen_US
dc.description.versionAccepted manuscripten_US
dash.depositing.authorShieber, Stuart
dc.date.available2019-07-03T14:25:41Z
dash.affiliation.otherHarvard John A. Paulson School of Engineering and Applied Sciencesen_US
dash.contributor.affiliatedGehrmann, Sebastian
dash.contributor.affiliatedSuzgun, Mirac
dash.contributor.affiliatedBelinkov, Yonatan
dash.contributor.affiliatedShieber, Stuart


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record