Supplemental Information: Readability Assessment and Comparison of Large Language Model-Generated Summaries of Trial Descriptions on ClinicalTrials.gov
Year: 2024
Type: supplementary-materials
Abstract: This study evaluated the readability of ClinicalTrials.gov trial information using traditional readability measures (TRMs) and compared it to summaries generated by large language models (LLMs), specifically ChatGPT and a fine-tuned BART-Large-CNN (FBLC). The study involved: 1) assessing required reading levels (RRL) with TRMs, 2) generating LLM-based summaries for a sample, and 3) evaluating summary quality through content and ranking scores averaged by two reviewers. Results showed that original summaries scored above the recommended readability level and did not improve over time. ChatGPT-generated summaries, however, had significantly lower RRLs and higher quality scores compared to original and FBLC summaries. High RRLs in ClinicalTrials.gov summaries may hinder participant recruitment and consent. ChatGPT shows promise for creating readable, high-quality summaries, and future research will evaluate ChatGPT-generated summaries and improve readability assessment metrics.
Authors: Wu, Tzu-Chun, Shih, Hanniel, Nattam, Anunita, Chintalapalli, Himaja, Hanauer, David, Zheng, Kai, Wu, Danny T.Y.
Institutions: University of Cincinnati, University of California, Irvine
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6930467800
Cites:
Cited by:
Related to: