Spectrum-BERT: Pretraining of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors
Year: 2024
Type: article
Abstract: Counterfeit Chinese liquor incidents in China have significantly disrupted market order and jeopardized the health of consumers. Currently, deep learning-based spectral detection techniques are extensively leveraged in non-invasive food inspection. Excessive reliance on labels severely limits its application in real scenarios. To make better use of limited samples, we are the first to use the “unsupervised pre-training & supervised fine-tuning” paradigm in combining the Transformer architecture for feature extraction and classification of the Chinese liquor spectrum, and propose Spectrum-BERT, which represents Bidirectional Encoder Representations from Transformers for Spectrum. Specifically, we creatively propose spectral curve partitioning and 1-D convolutional layer mapping to maintain the model’s sensitivity to characteristic peak locations and local information of spectral curves. Moreover, the paradigm of “unsupervised pre-training & supervised fine-tuning” addresses the limitation of label deficiency, thus improving the model’s applicability. Finally, we have conducted extensive experiments on the real liquor spectral dataset. Comparative experiments demonstrate that Spectrum-BERT outperforms all baselines on all metrics using only 70% supervised signal. The limit experimental results show that Spectrum-BERT can still maintain its lead using only the 10% supervised signal. Thanks to the more efficient model architecture, Spectrum-BERT’s model parameters and FLOPs are only 1/568 and 1/322 of those of baselines.
Institution: Harbin Institute of Technology
Language: English
Subfield: Analytical Chemistry
Field: Chemistry
Domain: Physical Sciences
Open Access status: green
Awards: G2939507312, G4443896572, G7354447749
OpenAlex ID: W4392567331