OptiMatLLM: A Benchmark Suite and a Lightweight Domain-Specific LLM for Optical Materials Research
Year: 2026
Type: preprint
Abstract: The design of optical materials relies on highly specialized knowledge, which creates a major bottleneck for data-driven scientific discovery in the optical material domain. Large language models open new possibilities for scientific research, but general-purpose models struggle with the domainspecific physical logic and terminology. More importantly, no standardized evaluation benchmark exists to systematically assess how well LLMs perform on optical materials tasks. To fill this gap, we first construct two evaluation benchmarks, Optic_Choice and Optic_QA, designed specifically for the optical materials domain. We build a high-quality instruction-tuning dataset of 23,225 question-answer pairs through an automated multi-stage pipeline from a curated corpus of papers and textbook. We then develop OptiMatLLM, a 7-billion-parameter LLM fine-tuned LLM on this corpus. Evaluation results show that OptiMatLLM achieves top performance on our optical benchmarks, reaching 0.79 accuracy on Optic_Choice, while also performing competitively on broader scientific tests such as CMHChem, ChemBench, and MaScQA. The model retains strong general capabilities as well, scoring 0.91 on the GSM8K mathematical reasoning benchmark. We further provide a systematic comparison of multiple leading models on these benchmarks, establishing a reference for future work. All benchmarks, dataset, and model are publicly released to support further research on LLMs for optical and optoelectronic materials design.
Source: ChemRxiv
Authors: Xinyi Wang, Youle Wang, Lei Zhang
Language: English
Subfield: Materials Chemistry
Field: Materials Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7208778033