Skapande av ett Riktmärke för Semantisk Likhet : En Strategi för Evaluering av Semantisk Likhet av Texter Genererade av Stora Språkmodeller
Year: 2025
Type: dissertation
Abstract: Evaluating semantic similarity between two distinct texts poses a significant challenge, particularly in the context of the vast quantities of text generated by Large Language Models. Accurately assessing whether generated text closely resembles original content is crucial, as failure to do so could contribute to the dissemination of misinformation and related issues. However, automating semantic evaluation is inherently complex, given that semantics requires substantial human interpretation and has historically been difficult to model. This thesis aims to develop a benchmark for measuring semantic similarity between texts by comparing various existing strategies alongside novel approaches. The objective is to determine whether these strategies can be employed independently or in combination to effectively evaluate semantic similarity. This was done by systematic evaluation of various strategies and their comparative results, leading to the design and implementation of a Siamese Neural Network. By utilizing Jaccard Similarity for regularization, a benchmark network was established to assess semantic similarity. The benchmark was initially trained on a specific task and subsequently tested on a downstream task involving longer reports. The results demonstrate that the network outperformed all other strategies across most metrics, and the transfer from the initial task to the downstream task was considered successful.
Author: Hammer, Rasmus
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6996627654