Work

Reliability and Readability of AI Chatbot Responses on Venous Thromboembolism After Fracture Surgery: A Comparative Study

Year: 2026
Type: preprint
Abstract: Background Venous thromboembolism (VTE) is an important complication after fracture surgery, and patients increasingly seek health information through internet based resources and artificial intelligence (AI) chatbots. However, the reliability and readability of AI generated information about postoperative VTE remain unclear. This study evaluated public information needs regarding VTE and assessed the reliability and readability of information provided by four widely used AI chatbots. Methods Google Trends was used to characterize worldwide public search interest in VTE from August 2021 to August 2026. Twenty patient oriented questions were developed based on popular searches and clinical experience and submitted verbatim to ChatGPT-5.6 Sol, Claude Sonnet 5, DeepSeek-V4-Pro, and Gemini 3.6 Flash. Reliability was assessed using DISCERN, EQIP, GQS, and JAMA criteria, while readability was evaluated using six established indices. Results Global search interest in VTE increased markedly from mid 2025 and peaked in July 2026. Reliability differed significantly among the four chatbots across all four assessment tools, with ChatGPT showing the highest median DISCERN, EQIP, and JAMA scores; all omnibus comparisons remained significant after Holm–Šídák correction (adjusted P < 0.0001). Readability remained poor across all models, with grade level indices well above the recommended sixth grade benchmark and FRES values ranging from 28.00 to 36.35, far below the recommended threshold of 80. Conclusion Public interest in VTE has increased in recent years, and contemporary AI chatbots differ in the reliability of information provided about VTE after fracture surgery. ChatGPT showed higher reliability scores across several domains, whereas source transparency remained limited across models. Responses from all four chatbots were substantially more complex than the recommended reading level for patient education. Improvements in reliability, source transparency, and readability are needed to enhance the usefulness of AI generated information for patient education.
Language: English
Field: Medicine
Open Access status: green
OpenAlex ID: W7225899038
Cites: 23
Cited by:
Related to: 10