Get started
Keyword

BERT

Description: A transformer-based pretrained language model that learns contextual representations of text for natural language processing tasks.
Primary topic: Topic Modeling (24%)
OpenAlex ID: keywords/bert
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
2019 · Jacob Devlin, Ming-Wei Chang, et al. · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · 48,188
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
2019 · Nils Reimers, Iryna Gurevych · 12,592
PDF
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
2019 · Jinhyuk Lee, Wonjin Yoon, et al. · Bioinformatics · 7,175
PDF
On the Dangers of Stochastic Parrots
2021 · Emily M. Bender, Timnit Gebru, et al. · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · 6,842
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
2019 · Schmidt, Wilma Johanna, Grangel-González, Irlan, et al. · arXiv (Cornell University) · 4,652
PDF
ALBERT-Base v2 (albert_base_v2.tar.gz): preservation copy
2019 · Lan, Zhenzhong, Chen, Mingda, et al. · arXiv (Cornell University) · 4,306
PDF
SimCSE: Simple Contrastive Learning of Sentence Embeddings
2021 · Tianyu Gao, Xingcheng Yao, et al. · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 2,830
PDF
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
2021 · Ahmed Elnaggar, Michael Heinzinger, et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2,305
Parameter-Efficient Transfer Learning for NLP
2019 · Houlsby, Neil, Giurgiu, Andrei, et al. · arXiv (Cornell University) · 2,269
PDF
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
2021 · Yu Gu, Robert Tinn, et al. · ACM Transactions on Computing for Healthcare · 2,145
PDF
TinyBERT: Distilling BERT for Natural Language Understanding
2020 · Xiaoqi Jiao, Yichun Yin, et al. · 1,778
PDF
Publicly Available Clinical
2019 · Emily Alsentzer, John Murphy, et al. · Clinical Natural Language Processing Workshop (ClinicalNLP) · 1,721
A Primer in BERTology: What We Know About How BERT Works
2020 · Anna Rogers, Olga Kovaleva, et al. · Transactions of the Association for Computational Linguistics · 1,544
PDF
How to Fine-Tune BERT for Text Classification?
2019 · Chi Sun, Xipeng Qiu, et al. · Lecture notes in computer science · 1,523
ERNIE: Enhanced Language Representation with Informative Entities
2019 · Zhengyan Zhang, Xu Han, et al. · Annual Meeting of the Association for Computational Linguistics (ACL) · 1,479
PDF
What Does BERT Look at? An Analysis of BERT’s Attention
2019 · Kevin Clark, Urvashi Khandelwal, et al. · 1,405
PDF
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
2023 · Bonan Min, Hayley Ross, et al. · ACM Computing Surveys · 1,375
PDF
BERT Rediscovers the Classical NLP Pipeline
2019 · Ian Tenney, Dipanjan Das, et al. · Annual Meeting of the Association for Computational Linguistics (ACL) · 1,304
PDF
What Does BERT Learn about the Structure of Language?
2019 · Ganesh Jawahar, Benoît Sagot, et al. · Annual Meeting of the Association for Computational Linguistics (ACL) · 1,288
PDF
ColBERT
2020 · Omar Khattab, Matei Zaharia · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · 1,253
VideoBERT: A Joint Model for Video and Language Representation Learning
2019 · Chen Sun, Austin Myers, et al. · IEEE/CVF International Conference on Computer Vision (ICCV) · 1,206
PDF
Multi-Task Deep Neural Networks for Natural Language Understanding
2019 · Xiaodong Liu, Pengcheng He, et al. · Annual Meeting of the Association for Computational Linguistics (ACL) · 1,184
PDF
Deberta: Decoding-enhanced Bert With Disentangled Attention
2021 · Pengcheng He, Xiaodong Liu, et al. · International Conference on Learning Representations · 1,082
LEGAL-BERT: The Muppets straight out of Law School
2020 · Ilias Chalkidis, Manos Fergadiotis, et al. · 1,060
PDF
BLEURT: Learning Robust Metrics for Text Generation
2020 · Thibault Sellam, Dipanjan Das, et al. · Annual Meeting of the Association for Computational Linguistics (ACL) · 1,001
PDF
Works count: 30,380
Citation count: 374,100
Year
Topic
Type