Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
inter-annotator reliability
Description:
Inter-annotator reliability is the degree to which multiple annotators consistently assign the same labels or judgments, assessed in data annotation research.
Related topics:
Sentiment Analysis and Opinion Mining (10%),
Natural Language Processing Techniques (8%),
Reliability and Agreement in Measurement (8%)
OpenAlex ID:
keywords/inter-annotator-reliability
Inter-Coder Agreement for Computational Linguistics
2008
·
Ron Artstein
, Massimo Poesio
·
Computational Linguistics
·
2,492
PDF
The Kappa Statistic: A Second Look
2004
·
Barbara Di Eugenio
, Michael Glass
·
Computational Linguistics
·
576
PDF
The ICSI Meeting Recorder Dialog Act (MRDA) Corpus
2004
·
Elizabeth Shriberg
, Raj Dhillon
, et al.
·
Defense Technical Information Center (DTIC)
·
387
Understanding Emotions: A Dataset of Tweets to Study Interactions between Affect Categories
2018
·
Saif Mohammad
, Svetlana Kiritchenko
·
Language Resources and Evaluation Conference (LREC)
·
137
PDF
Computing Reliability for Coreference Annotation
2004
·
Rebecca J. Passonneau
·
Language Resources and Evaluation Conference (LREC)
·
121
PDF
The alternatives (alt) tier for toBI: advantages of capturing prosodic ambiguity
2008
·
Alejna Brugos
, Nanette Veilleux
, et al.
·
Speech prosody
·
118
PDF
Vocal development in a large‐scale crosslinguistic corpus
2021
·
Margaret Cychosz
, Alejandrina Cristia
, et al.
·
Developmental Science
·
66
An agreement measure for determining inter-annotator reliability of human judgements on affective text
2008
·
Plaban Kr. Bhowmick
, Pabitra Mitra
, et al.
·
63
Speech pause distribution as an early marker for Alzheimer’s disease
2021
·
Patricia Pastoriza-Domínguez
, Iván G. Torre
, et al.
·
Speech Communication
·
62
Toward Fine-grained Annotation of Modality in Text
2013
·
Aynat Rubinstein
, Hillary Harner
, et al.
·
48
Towards a metaphor-annotated corpus of Mandarin Chinese
2017
·
Xiaofei Lu
, Ben Pin-Yun Wang
·
Language Resources and Evaluation
·
43
Crowdsourcing for web genre annotation
2016
·
Noushin Rezapour Asheghi
, Serge Sharoff
, et al.
·
Language Resources and Evaluation
·
34
PDF
Automatic TIMEX2 tagging of Korean news
2004
·
Seok Bae Jang
, Jennifer Baldwin
, et al.
·
ACM Transactions on Asian Language Information Processing
·
29
When the Prompt becomes the Codebook: Grounded Prompt Engineering (GROPROE) and its application to Belonging Analytics
2025
·
Sriram Ramanathan
, Lisa-Angelique Lim
, et al.
·
International Conference on Learning Analytics and Knowledge (LAK)
·
26
An experimental study measuring human annotator categorization agreement on commonsense sentences
2021
·
Henrique Santos
, Mayank Kejriwal
, et al.
·
Experimental Results
·
21
PDF
Inter-annotator reliability of medical events, coreferences and temporal relations in clinical narratives by annotators with varying levels of clinical expertise.
2012
·
Preethi Raghavan
, Eric Fosler-Lussier
, et al.
·
PubMed
·
21
How Far Are We from Genuinely Useful Deep Research Agents?
2025
·
Zhang, Dingling
, Zhu, He
, et al.
·
arXiv (Cornell University)
·
19
PDF
A study of terminology auditors’ performance for UMLS semantic type assignments
2012
·
Huanying (Helen) Gu
, Gai Elhanan
, et al.
·
Journal of Biomedical Informatics
·
18
PDF
A usage-based approach to metaphor identification and analysis in child speech
2023
·
Dorota Gaskins
, Marianna Falcone
, et al.
·
Language and Cognition
·
17
PDF
PERC-An Emotion Recognition Corpus for Cognitive Poems
2019
·
Sreeja P. S.
, G. S. Mahalakshmi
·
International Conference on Communications and Signal Processing (ICCSP)
·
12
FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Models
2026
·
Masoomali Fatehkia
, Enes Altinisik
, et al.
·
Conference of the European Chapter of the Association for Computational Linguistics
·
10
PDF
Fine-grained Emotion Detection in Microblog Text
2016
·
Jasy Liew Suet Yan
·
Syracuse University Libraries (Syracuse University)
·
10
Annotating Article Errors in Spanish Learner Texts: Design and Evaluation of an Annotation Scheme
2014
·
M. Pilar Valverde-Ibanez
, Akira Ohtani
·
Waseda University Repository (Waseda University)
·
10
PDF
Annotating argumentation in Swedish social media
2020
·
Anna Lindahl
·
8
Meeting acts: a labeling system for group interaction in meetings
2005
·
Rebecca Bates
, Patrick Menning
, et al.
·
Interspeech (USB)
·
8
Show more works
View as search
Works count:
116
Citation count:
4,739
Year
Topic
Type