Year: 2018
Type: conference-paper
Abstract: Many natural and formal languages contain words or symbols that require a matching counterpart for making an expression wellformed.The combination of opening and closing brackets is a typical example of such a construction.Due to their commonness, the ability to follow such rules is important for language modeling.Currently, recurrent neural networks (RNNs) are extensively used for this task.We investigate whether they are capable of learning the rules of opening and closing brackets by applying them to synthetic Dyck languages that consist of different types of brackets.We provide an analysis of the statistical properties of these languages as a baseline and show strengths and limits of Elman-RNNs, GRUs and LSTMs in experiments on random samples of these languages.In terms of perplexity and prediction accuracy, the RNNs get close to the theoretical baseline in most cases.
DOI: 10.18653/v1/w18-5425
Institution: Saarland University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: Deutsche Forschungsgemeinschaft
Award: G1951765764
OpenAlex ID: W2914557243