FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models
Year: 2023
Type: conference-paper
Abstract: It is common sense that one should prefer to eat a salad with a fork rather than with a chainsaw.However, for eating a bowl of rice, the choice between a fork and a pair of chopsticks is culturally relative.We introduce FORK, a small, manually-curated set of CommonsenseQA-style questions for probing cultural biases and assumptions present in commonsense reasoning systems, with a specific focus on food-related customs.We test several CommonsenseQA systems on FORK, and while we see high performance on questions about the US culture, the poor performance of these systems on questions about non-US cultures highlights systematic cultural biases aligned with US over non-US cultures.
Authors: Shramay Palta, Rachel Rudinger
Institution: University of Maryland, College Park
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4385572212