Year: 2023
Type: preprint
Abstract: African languages are severely under-represented in NLP research due to lack of datasets covering several NLP tasks. While there are individual language specific datasets that are being expanded to different tasks, only a handful of NLP tasks (e.g. named entity recognition and machine translation) have standardized benchmark datasets covering several geographical and typologically-diverse African languages. In this paper, we develop MasakhaNEWS -- a new benchmark dataset for news topic classification covering 16 languages widely spoken in Africa. We provide an evaluation of baseline models by training classical machine learning models and fine-tuning several language models. Furthermore, we explore several alternatives to full fine-tuning of language models that are better suited for zero-shot and few-shot learning such as cross-lingual parameter-efficient fine-tuning (like MAD-X), pattern exploiting training (PET), prompting language models (like ChatGPT), and prompt-free sentence transformer fine-tuning (SetFit and Cohere Embedding API). Our evaluation in zero-shot setting shows the potential of prompting ChatGPT for news topic classification in low-resource African languages, achieving an average performance of 70 F1 points without leveraging additional supervision like MAD-X. In few-shot setting, we show that with as little as 10 examples per label, we achieved more than 90\% (i.e. 86.0 F1 points) of the performance of full supervised training (92.6 F1 points) leveraging the PET approach.
Source: arXiv (Cornell University)
Authors: Adelani, David Ifeoluwa, Masiak, Marek, Azime, Israel Abebe, Alabi, Jesujoba, Tonja, Atnafu Lambebo, Mwase, Christine, Ogundepo, Odunayo, Dossou, Bonaventure F. P., Oladipo, Akintunde, Nixdorf, Doreen, Emezue, Chris Chinenye, al-azzawi, sana, Sibanda, Blessing, David, Davis, Ndolela, Lolwethu, Mukiibi, Jonathan, Ajayi, Tunde, Moteu, Tatiana, Odhiambo, Brian, Owodunni, Abraham, Obiefuna, Nnaemeka, Mohamed, Muhidin, Muhammad, Shamsuddeen Hassan, Ababu, Teshome Mulugeta, Salahudeen, Saheed Abdullahi, Yigezu, Mesay Gemeda, Gwadabe, Tajuddeen, Abdulmumin, Idris, Taye, Mahlet, Awoyomi, Oluwabusayo, Shode, Iyanuoluwa, Adelani, Tolulope, Abdulganiyu, Habiba, Omotayo, Abdul-Hakeem, Adeeko, Adetola, Afolabi, Abeeb, Aremu, Anuoluwapo, Samuel, Olanrewaju, Siro, Clemencia, Kimotho, Wangari, Ogbu, Onyekachi, Mbonu, Chinedu, Chukwuneke, Chiamaka, Fanijo, Samuel, Ojo, Jessica, Awosan, Oyinkansola, Kebede, Tadesse, Sakayo, Toadoum Sari, Nyatsine, Pamela, Sidume, Freedmore, Yousuf, Oreen, Oduwole, Mardiyyah, Tshinu, Tshinu, Kimanuka, Ussen, Diko, Thina, Nxakama, Siyanda, Nigusse, Sinodos, Johar, Abdulmejid, Mohamed, Shafie, Hassan, Fuad Mire, Mehamed, Moges Ahmed, Ngabire, Evrard, Jules, Jules, Ssenkungu, Ivan, Stenetorp, Pontus
Institutions: University College London, Saarland University, Instituto Politécnico Nacional, Fudan University, University of Waterloo, Technical University of Munich, Luleå University of Technology, Makerere University, Insight SFI Research Centre for Data Analytics, Aston University, Universidade do Porto, Dire Dawa University, Ahmadu Bello University, College of Saint Rose, Montclair State University, University of California, Davis, University of Rwanda, University of Amsterdam, African Institute for Mathematical Sciences, Nnamdi Azikiwe University, Lancaster University, Iowa State University, Botswana International University of Science and Technology, National Open University of Nigeria, Jamhuriya University of Science and Technology, Pan African University Institute for Basic Sciences, Technology and Innovation, Université du Burundi
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: DeepMind
OpenAlex ID: W4366732702