Dissecting the Frog: Understanding ChatGPT's Sense of Humour
Year: 2025
Type: conference-paper
Abstract: As the popularity surrounding large language models like OpenAI’s GPT series grows, it is crucial to explore areas where they still underperform, particularly in understanding humour. While models such as ChatGPT have demonstrated state-of-the-art results on benchmark tests (Brown et al., 2020), studies like Borji (2023) highlight their consistent failures in explaining jokes. This research investigates why such failures occur by analysing ChatGPT’s ability to interpret humour using the General Theory of Verbal Humour (GTVH) (Attardo & Raskin, 1991). GTVH suggests that jokes arise from the interplay of two semantically overlapping but fundamentally opposed scripts. This study hypothesises that ChatGPT’s architecture would be able to identify these opposed scripts. To test this, ChatGPT was asked to analyse 14 variants of two novel jokes, each crafted according to GTVH’s seven differences. Each version was presented five times, and the responses were qualitatively analysed for patterns in success and failure. The findings showed that ChatGPT consistently failed to identify the relevant scripts and often misallocated importance, frequently overemphasising noun phrases or relying on familiar patterns from its training data. These errors mirror those seen in tasks requiring complex semantic reasoning, such as the Adversarial NLI dataset (Nie et al., 2020). There were further difficulties in spatial and physical reasoning, consistent with Borji (2023). Thus, the hypothesis was falsified. However, the study successfully identified specific shortcomings of ChatGPT, which may inform future work on the model.
Author: Hayden, Matt
Institution: Newcastle University
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7102398010
FWCI:
Cites:
Cited by:
Related to: