Year: 2025
Type: dissertation
Abstract: This thesis investigates whether a large language model can distinguish between AI-generated and human-written texts by using perplexity—a metric that quantifies how predictable a piece of text is to the model itself. The core hypothesis is that AI-generated text, having been created by a language model, should appear more predictable (lower perplexity), while human writing is generally less predictable (higher perplexity). To test this, we use an open-source language model to evaluate both human and AI-generated samples from a range of curated datasets. We then apply a statistical classifier to see how well perplexity alone can separate the two classes.Our findings show that perplexity works well as a detection signal in controlled scenarios—especially when both the generation and evaluation are performed by the same model and within the same style or domain. However, the separation between AI and human writing becomes less clear when we introduce more varied sources, revealing the limitations of relying on perplexity alone.
Authors: Lindberg, Karl, Nobel, Isak
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7026586564