Year: 2021
Type: conference-paper
Abstract: Recently, it has been found that monolingual English language models can be used as knowledge bases.Instead of structural knowledge base queries, masked sentences such as "Paris is the capital of [MASK]" are used as probes.We translate the established benchmarks TREx and GoogleRE into 53 languages.Working with mBERT, we investigate three questions.(i) Can mBERT be used as a multilingual knowledge base?Most prior work only considers English.Extending research to multiple languages is important for diversity and accessibility.(ii) Is mBERT's performance as knowledge base language-independent or does it vary from language to language?(iii) A multilingual model is trained on more text, e.g., mBERT is trained on 104 Wikipedias.Can mBERT leverage this for better performance?We find that using mBERT as a knowledge base yields varying performance across languages and pooling predictions across languages improves performance.Conversely, mBERT exhibits a language bias; e.g., when queried in Italian, it tends to predict Italy as the country of origin.
Authors: Nora Kassner, Philipp Dufter, Hinrich Schütze
Institution: Ludwig-Maximilians-Universität München
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W3127729136