Keyword

pre-training objectives

Description: Learning goals or prediction tasks that define what a model is trained to learn during its initial pre-training phase.
Primary topic: Topic Modeling (39%)
OpenAlex ID: keywords/pre-training-objectives
2019 · Raffel, Colin, Shazeer, Noam, et al. · arXiv (Cornell University) · 31,402
2024 · Jingyi Zhang, Jiaxing Huang, et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2,051
2021 · Xu Han, Zhengyan Zhang, et al. · AI Open · 1,338
2022 · Tay, Yi, Dehghani, Mostafa, et al. · arXiv (Cornell University) · 594
2024 · Rajvardhan Patil, And Venkat Gudivada · Applied Sciences · 467
2023 · Fei-Long Chen, Du-Zhen Zhang, et al. · Machine Intelligence Research · 428
2023 · Xiao Wang, Guangyao Chen, et al. · Machine Intelligence Research · 367
2020 · Alon Talmor, Yanai Elazar, et al. · Transactions of the Association for Computational Linguistics · 359
2023 · Xinyi Wang, Wanrong Zhu, et al. · neural information processing systems · 310
2023 · Ankita Pasad, Bowen Shi, et al. · IEEE International Conference on Acoustics Speech and Signal Processing · 297
2026 · Quanjun Zhang, Chunrong Fang, et al. · Science China Information Sciences · 274
2018 · Kelly Zhang, Samuel Bowman · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · 212
2021 · Roziere, Baptiste, Lachaux, Marie-Anne, et al. · arXiv (Cornell University) · 175
2019 · Alex Wang, Jan Hula, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 137
2022 · Zewen Chi, Shaohan Huang, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 123
2021 · Xinyu Ma, Jiafeng Guo, et al. · ACM International Conference on Web Search and Data Mining (WSDM) · 122
2024 · Ankita Pasad, Chung-Ming Chien, et al. · Transactions of the Association for Computational Linguistics · 102
2019 · Shikib Mehri, Evgeniia Razumovskaia, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 97
2022 · David Wan, Mohit Bansal · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · 94
2024 · Afnan Sultan, Jochen Sieg, et al. · Journal of Chemical Information and Modeling · 90
2024 · Zhongxin Liu, Zhijie Tang, et al. · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · 76
2023 · Honglu Zhou, Roberto Martín-Martín, et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 76
2022 · Rubachev, Ivan, Alekberov, Artem, et al. · arXiv (Cornell University) · 67
2026 · Yihao Ding, Soyeon Caren Han, et al. · Artificial Intelligence Review · 65
Works count: 287
Citation count: 41,410
Year
Topic
Type