Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
annotation reliability
Description:
Annotation reliability is the degree to which annotations are consistent and reproducible across annotators, tasks, or repeated evaluations.
Related topics:
Natural Language Processing Techniques (13%),
Sentiment Analysis and Opinion Mining (10%)
OpenAlex ID:
keywords/annotation-reliability
ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning
2023
·
Törnberg, Petter
·
arXiv (Cornell University)
·
552
PDF
Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis
2016
·
Roß, Björn
, Rist, Michael
, et al.
·
arXiv (Cornell University)
·
483
PDF
How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation
2010
·
Stefanie Nowak
, Stefan Rüger
·
Multimedia Information Retrieval
·
434
Metaphor in usage
2010
·
Gerard J. Steen
, Aletta G. Dorst
, et al.
·
Cognitive Linguistics
·
373
InFoBench: Evaluating Instruction Following Ability in Large Language Models
2024
·
Yiwei Qin
, Kaiqiang Song
, et al.
·
Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
·
342
PDF
The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena
2007
·
Jens Allwood
, Loredana Cerrato
, et al.
·
Computers and the Humanities
·
327
Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation
2017
·
Svetlana Kiritchenko
, Saif Mohammad
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
297
PDF
Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation
2006
·
Rebecca Passonneau
·
Language Resources and Evaluation Conference (LREC)
·
246
PDF
Experiences from the Spoken Dutch Corpus Project
2002
·
Nelleke Oostdijk
, Wim Goedertier
, et al.
·
Language Resources and Evaluation Conference (LREC)
·
190
PDF
The reliability of anaphoric annotation, reconsidered: taking ambiguity into account
2005
·
Massimo Poesio
, Ron Artstein
·
161
PANNZER: high-throughput functional annotation of uncharacterized proteins in an error-prone environment
2015
·
Patrik Koskinen
, Petri Törönen
, et al.
·
Bioinformatics
·
160
Learning part-of-speech taggers with inter-annotator agreement loss
2014
·
Barbara Plank
, Dirk Hovy
, et al.
·
Conference of the European Chapter of the Association for Computational Linguistics
·
153
The Unified and Holistic Method Gamma (γ) for Inter-Annotator Agreement Measure and Alignment
2015
·
Yann Mathet
, Antoine Widlöcher
, et al.
·
Computational Linguistics
·
150
PDF
The Ordinal Nature of Emotions: An Emerging Approach
2018
·
Georgios N. Yannakakis
, Roddy Cowie
, et al.
·
IEEE Transactions on Affective Computing
·
149
Fuzzy Measures on the Gene Ontology for Gene Product Similarity
2006
·
M. Popescu
, J.M. Keller
, et al.
·
IEEE Transactions on Computational Biology and Bioinformatics
·
148
A stepwise method for annotating appraisal
2018
·
Matteo Fuoli
·
Functions of Language
·
147
Community annotation experiment for ground truth generation for the i2b2 medication challenge
2010
·
Özlem Uzuner
, Imre Solti
, et al.
·
Journal of the American Medical Informatics Association
·
142
PDF
Annotation, Modelling and Analysis of Fine-Grained Emotions on a Stance and Sentiment Detection Corpus
2017
·
Hendrik Schuff
, Jeremy Barnes
, et al.
·
Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA)
·
137
PDF
Dimensional Modeling of Emotions in Text with Appraisal Theories: Corpus Creation, Annotation Reliability, and Prediction
2022
·
Enrica Troiano
, Laura Oberländer
, et al.
·
Computational Linguistics
·
131
Automatic annotation of protein function
2005
·
Alfonso Valencia
·
Current Opinion in Structural Biology
·
131
Quality of Computationally Inferred Gene Ontology Annotations
2012
·
Nives Škunca
, Adrian Altenhoff
, et al.
·
PLoS Computational Biology
·
130
PDF
On Reliability of Patch Correctness Assessment
2019
·
Xuan-Bach D. Le
, Lingfeng Bao
, et al.
·
ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)
·
121
PDF
Evaluating large language models in analysing classroom dialogue
2024
·
Yun Long
, Haifeng Luo
, et al.
·
npj Science of Learning
·
115
PDF
Adding Noun Phrase Structure to the Penn Treebank
2007
·
David Vadas
, James Curran
·
108
Automated Gene Ontology annotation for anonymous sequence data
2003
·
S. Hennig
, Detlef Groth
, et al.
·
Nucleic Acids Research
·
107
Show more works
View as search
Works count:
608
Citation count:
8,418
Year
Topic
Type