Evaluating the Diagnostic Potential of Large Language Models in Fetal Alcohol Spectrum Disorder
Year: 2024
Type: dissertation
Abstract: Large language models enjoy wide spread applications in both general and more personalizeduse cases. These models can be dynamically trained on well defined clinical data. However,\nseveral pre-existing models that have not been trained to provide diagnostic information\nfor disorders with clinical heterogeneity. Specifically, our preliminary analysis showed that\nexisting models such as BioMistral are generalized on publicly available PubMed data but are\nunable to accurately take in clinical symptoms for accurate characterization of fetal alcohol\nspectrum disorder (FASD). To overcome this challenge, we propose to retrain the pre-existing\nBioMistral model on a synthetic FASD-specific training set to correctly categorize symptoms\ninto diagnostic codes. By changing the learning rates and epochs, we are able to evaluate\nthe performance of both overfitted or poorly trained models and a highly trained model on\na test set containing synthetic clinical notes. We demonstrate evaluation performance using\nconfusion matrices and the Kullback-leibler divergence (on the log-odds probabilities) and\nshow that retraining BioMistral model has the capability to correctly diagnose individuals\nwith fetal alcohol spectrum disorder over a poorly trained model.
Author: Yuan, Faith Tsyr-Huey
Institution: University of California, Los Angeles
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6981698453