Keyword

language model pretraining

Description: Language model pretraining is the initial learning stage in which a model acquires language patterns from data before task-specific adaptation or use.
Primary topic: Topic Modeling (55%)
OpenAlex ID: keywords/language-model-pretraining
2019 · Liu, Yinhan, Ott, Myle, et al. · arXiv (Cornell University) · 40,948
2020 · Kelvin Guu, Kenton Lee, et al. · International Conference on Machine Learning · 2,476
2019 · Chi Sun, Xipeng Qiu, et al. · Lecture notes in computer science · 2,472
2019 · Telmo Pires, Eva Schlinger, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 2,355
2022 · Zhengxiao Du, Yujie Qian, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 1,917
2020 · Francesco Barbieri, Jose Camacho-Collados, et al. · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · 1,414
2020 · Guu, Kelvin, Lee, Kenton, et al. · arXiv (Cornell University) · 1,330
2020 · Allyson Ettinger · Transactions of the Association for Computational Linguistics · 808
2024 · Luca Soldaini, Rodney Kinney, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 672
2022 · Michihiro Yasunaga, Jure Leskovec, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 615
2018 · Phang, Jason, Févry, Thibault, et al. · arXiv (Cornell University) · 582
2023 · Sang Michael Xie, Hieu Pham, et al. · neural information processing systems · 491
2023 · Xia, Mengzhou, Gao, Tianyu, et al. · arXiv (Cornell University) · 454
2020 · Kaj Bostrom, Greg Durrett · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · 452
2021 · Du, Zhengxiao, Qian, Yujie, et al. · arXiv (Cornell University) · 450
2022 · Luyu Gao, Jamie Callan · Proceedings of the conference - Association for Computational Linguistics. Meeting · 434
2023 · Sang Michael Xie, Shibani Santurkar, et al. · neural information processing systems · 414
2022 · Abhik Bhattacharjee, Tahmid Hasan, et al. · Findings of the Association for Computational Linguistics: NAACL · 366
2023 · Alex Warstadt, Aaron Mueller, et al. · BabyLM Challenge · 363
2023 · Liu, Hong, Li, Zhiyuan, et al. · arXiv (Cornell University) · 353
2023 · Korbak, Tomasz, Shi, Kejian, et al. · arXiv (Cornell University) · 347
2020 · Jiezhong Qiu, Hao Ma, et al. · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · 304
2021 · Oshin Agarwal, Heming Ge, et al. · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · 296
2024 · Gerstgrasser, Matthias, Schaeffer, Rylan, et al. · arXiv (Cornell University) · 283
2021 · Liu, Qian, Chen, Bei, et al. · arXiv (Cornell University) · 280
Works count: 613
Citation count: 70,040
Year
Topic
Type