Year: 2024
Type: dissertation
Abstract: Today's large language models (LLMs) are extremely good at generating text practically indistinguishable from the human one, but are prone to hallucinate information. Therefore, there is a need for fact-checking, particularly in sensitive and critical domains where misinformation could have life-threatening consequences, as seen in the medical field. This task presents a significant challenge due to the exceptional quality of the artificial text. Moreover, developing highly effective fact-checking tools in specialized domains still needs to be solved in the literature. In this thesis, we introduce Med-vera, taking a step in automatically quantifying the plausibility of declarative medical statements. To better align with real-world scenarios, we focus on evidence-free fact-checking, which requires a statement as input without needing a trustworthy source paragraph to compare with. Using LLaMA-2-chat, Med-vera verbalizes existing heterogeneous medical resources to generate over 1 million correct and incorrect artificial statements, which serve as the basis for training closed-book classification models. The converted resources are question-answering datasets (BioASQ, MedMCQA, PubMedQA) and knowledge graphs (UMLS).
Author: Bedeschi, Federica
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7009099629