Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
Year: 2024
Type: conference-paper
Abstract: Existing research predominantly focuses on developing powerful large language models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in preserving efficacy in a multilingual context.To bridge this gap, this paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs.Firstly, by utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages, thus addressing the issue of training data scarcity in xMR tasks.Based on the collected dataset, we propose different training strategies to build powerful xMR LLMs, named MathOctopus, which notably outperform conventional open-source LLMs and exhibit superiority over ChatGPT in few-shot scenarios.Notably, MathOctopus-13B reaches 47.6% accuracy which exceeds ChatGPT 46.3% on MGSM testset.Beyond remarkable results, we unearth several pivotal observations and insights: (1) When extending the rejection sampling strategy to the multilingual context, it proves effective for model performances, albeit limited.(2) Employing parallel corpora for math Supervised Fine-Tuning (SFT) across multiple languages not only significantly enhances model performance multilingually and elevates their monolingual performance.This indicates that crafting multilingual corpora can be regarded as a vital strategy for enhancing model performance in a specific language, especially in mathematical reasoning tasks.For instance, MathOctopus-7B improves its counterparts that trained on English from 42.4% to 50.8% on the GSM8K test set.James decides to run 3 sprints 3 times a week.He runs 60 meters each sprint.How many total meters does he run a week?English 詹姆斯决定每周跑 3 次 3 段冲刺，每段冲刺跑 60 米。他 每周一共跑多少米？ Chinese James beschließt, 3-mal pro Woche 3 Kurzstreckenläufe zu absolvieren.Pro Kurzstreckenlauf läuft er 60 Meter.Wie viele Meter insgesamt läuft er pro Woche?German French Japanese …… He sprints 3*3=<<3*3=9>>9 time.So he runs 9*60=<<9*60 =540>>540 meters.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: hybrid
Award: G1402809924
OpenAlex ID: W4404781253