Work

Assessing the performance of ChatGPT's responses to questions related to epilepsy: A cross-sectional study on natural language processing and medical information retrieval

Year: 2023
Type: article
Abstract: Background Epilepsy is a neurological condition marked by frequent seizures and various cognitive and psychological effects. Reliable information is essential for effective treatment. Natural language processing models like ChatGPT are increasingly used in healthcare for information access and data analysis, making it crucial to assess their accuracy. Objective This study aimed to investigate the accuracy of ChatGPT in providing educational information related to epilepsy. Methods We compared the answers from ChatGPT-4 and ChatGPT-3.5 to 57 common epilepsy questions based on the Korean Epilepsy Society's "Epilepsy Patient and Caregiver Guide." Two epileptologists reviewed the responses, with a third serving as an arbiter in cases of disagreement. Results Out of 57 questions, 40 responses from ChatGPT-4 had "sufficient educational value," 16 were "correct but inadequate," and one was "mixed with correct and incorrect" information. No answers were entirely incorrect. GPT-4 generally outperformed GPT-3.5 and was often on par with or better than the official guide. Conclusions ChatGPT-4 shows promise as a tool for delivering reliable epilepsy-related information and could help alleviate the educational burden on healthcare professionals. Further research is needed to explore the benefits and limitations of using such models in medical contexts.
Source: Seizure
Language: English
Field: Medicine
Open Access status: hybrid
Estimated APC paid: $2,500
Award: G214355609
OpenAlex ID: W4388923875
FWCI: 2.157
Cites: 30
Cited by: 68
Related to: 8