SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research
Year: 2023
Type: preprint
Abstract: Despite its relevance, the maturity of NLP for social media pales in comparison with general-purpose models, metrics and benchmarks. This fragmented landscape makes it hard for the community to know, for instance, given a task, which is the best performing model and how it compares with others. To alleviate this issue, we introduce a unified benchmark for NLP evaluation in social media, SuperTweetEval, which includes a heterogeneous set of tasks and datasets combined, adapted and constructed from scratch. We benchmarked the performance of a wide range of models on SuperTweetEval and our results suggest that, despite the recent advances in language modelling, social media remains challenging.
Source: arXiv (Cornell University)
Institutions: Cardiff University, Snap (United States), University of Michigan
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: UK Research and Innovation
OpenAlex ID: W4387929426