Person:
Belinkov, Yonatan

Profile Picture

Email Address

AA Acceptance Date

Birth Date

Research Projects

Organizational Units

Job Title

Last Name

Belinkov

First Name

Yonatan

Name

Belinkov, Yonatan

Filters

20194

Settings

Author's Publications: search.filters.isAuthorOfPublication.fedeb170-1148-4403-b3b8-e5f7b77f6dbd

Search Results

Now showing 1 - 4 of 4
  • No Thumbnail Available
    Publication
    On Evaluating the Generalization of LSTM Models in Formal Languages
    (Society for Computation in Linguistics, 2019-01) Suzgun, Mirac; Belinkov, Yonatan; Shieber, Stuart
    Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing. Yet, there still remains an uncertainty regarding their language learning capabilities. In this paper, we empirically evaluate the inductive learning capabilities of Long Short-Term Memory networks, a popular extension of simple RNNs, to learn simple formal languages, in particular $a^nb^n$, $a^nb^nc^n$, and $a^n b^n c^n d^n$. We investigate the influence of various aspects of learning, such as training data regimes and model capacity, on the generalization to unobserved samples. We find striking differences in model performances under different training settings and highlight the need for careful analysis and assessment when making claims about the learning capabilities of neural network models.
  • Thumbnail Image
    Publication
    LSTM Networks Can Perform Dynamic Counting
    (2019-06-09) Suzgun, Mirac; Gehrmann, Sebastian; Belinkov, Yonatan; Shieber, Stuart
    In this paper, we systematically assess the ability of standard recurrent networks to perform dynamic counting and to encode hierarchical representations. All the neural models in our experiments are designed to be small-sized networks both to prevent them from memorizing the training sets and to visualize and interpret their behaviour at test time. Our results demonstrate that the Long Short-Term Memory (LSTM) networks can learn to recognize the well-balanced parenthesis language (Dyck-1) and the shuffles of multiple Dyck-1 languages, each defined over different parenthesis-pairs, by emulating simple real-time k-counter machines. To the best of our knowledge, this work is the first study to introduce the shuffle languages to analyze the computational power of neural networks. We also show that a single-layer LSTM with only one hidden unit is practically sufficient for recognizing the Dyck-1 language. However, none of our recurrent networks was able to yield a good performance on the Dyck-2 language learning task, which requires a model to have a stack-like mechanism for recognition.
  • Thumbnail Image
    Publication
    Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference
    (Association of Computational Linguistics, 2019-07) Belinkov, Yonatan; Poliak, Adam; Shieber, Stuart; Van Durme, Benjamin; Rush, Alexander Sasha
    Natural Language Inference (NLI) datasets often contain hypothesis-only biases—artifacts that allow models to achieve non-trivial performance without learning whether a premise entails a hypothesis. We propose two probabilistic methods to build models that are more robust to such biases and better transfer across datasets. In contrast to standard approaches to NLI, our methods predict the probability of a premise given a hypothesis and NLI label, discouraging models from ignoring the premise. We evaluate our methods on synthetic and existing NLI datasets by training on datasets containing biases and testing on datasets containing no (or different) hypothesis-only biases. Our results indicate that these methods can make NLI models more robust to dataset-specific artifacts, transferring better than a baseline architecture in 9 out of 12 NLI datasets. Additionally, we provide an extensive analysis of the interplay of our methods with known biases in NLI datasets, as well as the effects of encouraging models to ignore biases and fine-tuning on target datasets.
  • Thumbnail Image
    Publication
    On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference
    (2019-06) Belinkov, Yonatan; Poliak, Adam; Shieber, Stuart; Van Durme, Benjamin; Rush, Alexander Sasha
    Popular Natural Language Inference (NLI) datasets have been shown to be tainted by hypothesis-only biases. Adversarial learning may help models ignore sensitive biases and spurious correlations in data. We evaluate whether adversarial learning can be used in NLI to encourage models to learn representa- tions free of hypothesis-only biases. Our analyses indicate that the representations learned via adversarial learning may be less biased, with only small drops in NLI accuracy.