Year: 2026
Type: preprint
Abstract: Databases of thermoelectric properties are built largely by extracting values from the literature, and the choice of extraction method determines how many records are recovered and how many are wrong. We compare five methods on the same paragraph-level task: a corpus-informed regular-expression baseline, the published ChemDataExtractor thermoelectric parsers, a MaterialsBERT-based named-entity model with rule-based linking, a single-prompt large language model (LLM) and an LLM agent. The methods were developed on a 100-paragraph development sample (the rules of the regular-expression baseline earlier, on the whole corpus), frozen, and run on a 200-paragraph test sample drawn by stratified sampling from 311 papers (mostly arXiv preprints), with a paper-level split between the two samples. A record counts as correct only if property, value, temperature and material all match the gold standard. On the 160 in-scope test paragraphs (241 core records), the single-prompt LLM reached F1 = 0.79 (95 % CI 0.71–0.87) and the agent 0.73 (0.64–0.83), against 0.22 for the regular-expression baseline and 0.06 for the ChemDataExtractor and NER pipelines. Both LLM methods recover property, value and temperature with F1 above 0.94; most of their remaining losses arise from the material criterion; in most of these cases one material name contains the other. We found no evidence of an accuracy improvement from the tested agent configuration, which differs from the single prompt in several components at once. We release the protocol, code, gold standard and all method outputs in a public repository.
Source: ChemRxiv
Author: Berna Uyanık
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7223086249