Work

PhysBERT: A text embedding model for physics scientific literature

Year: 2024

Type: article

Abstract: The specialized language and complex concepts in physics pose significant challenges for information extraction through Natural Language Processing (NLP). Central to effective NLP applications is the text embedding model, which converts text into dense vector representations for efficient information retrieval and semantic analysis. In this work, we introduce PhysBERT, the first physics-specific text embedding model. Pre-trained on a curated corpus of 1.2 × 106 arXiv physics papers and fine-tuned with supervised data, PhysBERT outperforms leading general-purpose models on physics-specific tasks, including the effectiveness in fine-tuning for specific physics subdomains.

Source: APL Machine Learning

DOI: 10.1063/5.0238090

Authors: Thorsten Hellert, João Montenegro, Andrea Pollastro

Institutions: Lawrence Berkeley National Laboratory, University of Naples Federico II

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funders: U.S. Department of Energy, National Energy Research Scientific Computing Center, Office of Science