Work

LAPCA: Language-Agnostic Pretraining with Cross-Lingual Alignment

Year: 2023

Type: conference-paper

Abstract: Data collection and mining is a crucial bottleneck for cross-lingual information retrieval (CLIR). While previous works used machine translation and iterative training, we present a novel approach to cross-lingual pretraining called LAPCA (language-agnostic pretraining with cross-lingual alignment). We train the LAPCA-LM model based on XLM-RoBERTa and łexa that significantly improves cross-lingual knowledge transfer for question answering and sentence retrieval on, e.g., XOR-TyDi and Mr. TyDi datasets, and in the zero-shot cross-lingual scenario performs on par with supervised methods, outperforming many of them on MKQA.

Source: International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

DOI: 10.1145/3539618.3592006

Authors: Dmitry Abulkhanov, Nikita Sorokin, Sergey Nikolenko, Valentin Malykh

Institutions: Steklov Mathematical Institute, Institute for System Programming

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: Analytical Center for the Government of the Russian Federation, Russian Academy of Sciences

Awards: G2928192979, G434231340, G7129761866, G867576739