The RECUR corpus: annotated German radiology reports for urolithiasis research
Year: 2026
Type: article
Abstract: Background Advances in clinical NLP depend on high-quality datasets that reflect the complexity of real-world documentation. In response, a fine-grained, relation-annotated corpus for urinary stone diagnostics is introduced. The corpus addresses a gap in German medical NLP resources and provides a reference dataset for developing and evaluating clinical information extraction models. Methods We extracted and de-identified 262 radiology reports and 251 corresponding short findings concerning kidney and ureter stones from the hospital information system of the University Medical Center Freiburg. Using a detailed annotation schema comprising nine entity types and five relation types, three medical doctoral students annotated the corpus in Label Studio, followed by expert adjudication to establish a gold standard. For evaluation, we fine-tuned several German general-domain and medical language models for named entity recognition (NER) and relation extraction (RE) using a fixed train–development–test split, a unified Hugging Face pipeline, and NNI-based hyperparameter tuning. Results The resulting corpus comprises 513 radiology documents with 3,223 annotated entities, dominated by stone- and anatomy-related categories. Inter-annotator agreement was consistently high for both entity and relation annotations (NER F1 = 82.8%; RE F1 = 81.8%), supporting the reliability of the annotation schema. The evaluated models achieved macro F1-scores of up to 89.45% for NER and 99.24% for RE. While NER performance varied across models and fine-grained entity categories, RE performance was consistently high across all evaluated encoders. Conclusion RECUR establishes a fine-grained reference corpus and evaluation setting for automated information extraction from German radiology reports. Its adjudicated annotations, strong inter-annotator agreement, and comprehensive encoder baselines support the systematic evaluation of clinical NER and RE methods.
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,990
OpenAlex ID: W7236857631
