Year: 2014
Type: article
Abstract: Language identification is the task of automatically detecting the language(s) present in a document based on the content of the document. In this work, we address the problem of detecting documents that contain text from more than one language ( multilingual documents). We introduce a method that is able to detect that a document is multilingual, identify the languages present, and estimate their relative proportions. We demonstrate the effectiveness of our method over synthetic data, as well as real-world multilingual documents collected from the web.
DOI: 10.1162/tacl_a_00163
Authors: Marco Lui, Jey Han Lau, Timothy Baldwin
Institutions: The University of Melbourne, Data61, King's College London
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W2183864645