Outcomes of High CPRA, Bypassed or Refused Offer Candidates: The Missed Chance Might Be the Last
Year: 2025
Type: article
Abstract: deceaSed donor Kidney acceSS to WaitliSt and outcomeS S254presence of a subcapsular hematoma, cysts/discoloration, and if the fat was cleaned (N=7,490).We then sample approx.50% of these worksheets for evaluating LLM extraction accuracy.Outputs from this bespoke process are a reliable capture of the content contained in the renal worksheet and have been previously presented.We then render each renal worksheet page to an image and provide the page image as input to Qwen2-VL-7B (Qwen), an open-weight multi-modal LLM optimized for visually situated language comprehension.Included with the renal page is text that prompts the model to extract the measures of interest that we extracted using our bespoke approach.We then compare Qwen's output to the actual values extracted by our bespoke approach.Results: Our results, and an example input, are shown in Figure 1.With minimal prompt engineering nor any fine-tuning, we find that Qwen achieves promising, but inconsistent results.For example, Qwen demonstrates high recall and precision (F1 = 0.999) for "Fat Cleaned" across both lateralities.However, Qwen performs very poorly when extracting "Subcapsular Hematoma" for the right kidney (F1 = 0.504) yet achieves perfect performance for the left kidney (F1 = 1.0).Also, although our prompts suggest it as a possible response, Qwen never responded with "unknown".Conclusions: Our work demonstrates both the promises and perils of relying on LLMs for extracting information from unstructured deceased donor data.While Qwen achieves impressive performance in some cases, it still fails in others, suggesting that moderate optimizations for a transplant context may help Qwen reach acceptable reliability for completely automated extraction.Future work should evaluate additional open-weight models, fine-tuning for transplant, and applying these methods to extract novel insights from over a decade' s worth of unstructured attachment data.
Language: English
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Estimated APC paid: $4,300
OpenAlex ID: W4412850779