Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
BERT pre-training
Description:
The initial training of a BERT language model on large-scale data to learn contextual representations before adapting it to specific tasks.
Primary topic:
Topic Modeling (38%)
Related topics:
Topic Modeling (38%),
Sentiment Analysis and Opinion Mining (9%),
Natural Language Processing Techniques (7%)
OpenAlex ID:
keywords/bert-pre-training
SpanBERT: Improving Pre-training by Representing and Predicting Spans
2020
·
Mandar Joshi
, Danqi Chen
, et al.
·
Transactions of the Association for Computational Linguistics
·
2,546
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
2019
·
Baevski, Alexei
, Schneider, Steffen
, et al.
·
arXiv (Cornell University)
·
602
PDF
Order Matters: Semantic-Aware Neural Networks for Binary Code Similarity Detection
2020
·
Zeping Yu
, Rui Cao
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
315
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers
2023
·
Xiaoyi Dong
, Jianmin Bao
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
220
PDF
Chinese clinical named entity recognition with variant neural structures based on BERT methods
2020
·
Xiangyang Li
, Huan Zhang
, et al.
·
Journal of Biomedical Informatics
·
210
Neural Media Bias Detection Using Distant Supervision With BABE - Bias Annotations By Experts
2021
·
Timo Spinde
, Manuel Plank
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
153
PDF
Efficient Sequence Packing without Cross-contamination: Accelerating Large Language Models without Impacting Performance
2021
·
Krell, Mario Michael
, Kosec, Matej
, et al.
·
arXiv (Cornell University)
·
145
PDF
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers
2023
·
Dong, Xiaoyi
, Bao, Jianmin
, et al.
·
AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI))
·
141
MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education
2021
·
Shen, Jia Tracy
, Yamashita, Michiharu
, et al.
·
arXiv (Cornell University)
·
123
PDF
Multi-Modality Latent Interaction Network for Visual Question Answering
2019
·
Gao Peng
, Haoxuan You
, et al.
·
IEEE/CVF International Conference on Computer Vision (ICCV)
·
106
PDF
BEST: BERT Pre-training for Sign Language Recognition with Coupling Tokenization
2023
·
Weichao Zhao
, Hezhen Hu
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
100
PDF
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
2021
·
Tang, Hanlin
, Gan, Shaoduo
, et al.
·
arXiv (Cornell University)
·
91
PDF
Entity Enhanced BERT Pre-training for Chinese NER
2020
·
Chen Jia
, Yuefeng Shi
, et al.
·
Conference on Empirical Methods in Natural Language Processing (EMNLP)
·
83
PDF
Reducing BERT Pre-Training Time from 3 Days to 76 Minutes
2019
·
Yang You
, Jing Li
, et al.
·
arXiv (Cornell University)
·
80
Listen, Understand and Translate: Triple Supervision Decouples End-to-end Speech-to-text Translation
2021
·
Qianqian Dong
, Rong Ye
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
77
PERT: Pre-training BERT with Permuted Language Model
2022
·
Cui, Yiming
, Yang, Ziqing
, et al.
·
arXiv (Cornell University)
·
74
PDF
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
2022
·
Yixuan Su
, Fangyu Liu
, et al.
·
Findings of the Association for Computational Linguistics: NAACL
·
72
CAME: Confidence-guided Adaptive Memory Efficient Optimization
2023
·
Yang Luo
, Xiaozhe Ren
, et al.
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
65
PDF
DarijaBERT: a step forward in NLP for the written Moroccan dialect
2024
·
Kamel Gaanoun
, Abdou Mohamed Naira
, et al.
·
International Journal of Data Science and Analytics
·
64
PDF
WePos: Weak-supervised Indoor Positioning with Unlabeled WiFi for On-demand Delivery
2022
·
Baoshen Guo
, Weijian Zuo
, et al.
·
Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies
·
63
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-Text Retrieval
2022
·
Yuying Ge
, Yixiao Ge
, et al.
·
Lecture notes in computer science
·
57
PDF
L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Models
2022
·
Nayak, Ravindra
, Joshi, Raviraj
·
arXiv (Cornell University)
·
55
PDF
Automatic Graph Partitioning for Very Large-scale Deep Learning
2021
·
Masahiro Tanaka
, Kenjiro Taura
, et al.
·
IEEE International Parallel and Distributed Processing Symposium (IPDPS)
·
44
PDF
Generating Logical Forms from Graph Representations of Text and Entities
2019
·
Peter Shaw
, Philip Massey
, et al.
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
42
PDF
Graph-to-Graph Transformer for Transition-based Dependency Parsing
2020
·
Alireza Mohammadshahi
, James Henderson
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
40
PDF
Show more works
View as search
Works count:
153
Citation count:
6,518
Year
Topic
Type