Year: 2022
Type: conference-paper
Abstract: Weakly supervised named entity recognition methods train label models to aggregate the token annotations of multiple noisy labeling functions (LFs) without seeing any manually annotated labels. To work well, the label model needs to contextually identify and emphasize well-performed LFs while down-weighting the under-performers. However, evaluating the LFs is challenging due to the lack of ground truths. To address this issue, we propose the sparse conditional hidden Markov model (Sparse-CHMM). Instead of predicting the entire emission matrix as other HMM-based methods, Sparse-CHMM focuses on estimating its diagonal elements, which are considered as the reliability scores of the LFs. The sparse scores are then expanded to the full-fledged emission matrix with pre-defined expansion functions. We also augment the emission with weighted XOR scores, which track the probabilities of an LF observing incorrect entities. Sparse-CHMM is optimized through unsupervised learning with a three-stage training pipeline that reduces the training difficulty and prevents the model from falling into local optima. Compared with the baselines in the Wrench benchmark, Sparse-CHMM achieves a 3.01 average F1 score improvement on five comprehensive datasets. Experiments show that each component of Sparse-CHMM is effective, and the estimated LF reliabilities strongly correlate with true LF F1 scores.
Authors: Yinghao Li, Le Song, Chao Zhang
Institution: Georgia Institute of Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G11240557, III: Small: Go Beyond Short-term Dependency and Homogeneity: A General-Purpose Transformer Recipe for Multi-Domain Heterogeneous Sequential Data Analysis, G4504108201, G4558963220, CAREER: Accelerating Spatial Network Design: An Uncertainty-Driven Predict-and-Optimize Learning Framework, G5397617431, III: Medium: Collaborative Research: Principled Uncertainty Quantification in Deep Learning Models for Time Series Analysis, G6671297155, G8876996369
OpenAlex ID: W4281858832