Keyword

pre-training

Description: Pre-training is an initial learning phase in which a model or learner acquires general knowledge or skills before task-specific training or instruction.
Related topic: Topic Modeling (12%)
Wikidata ID: wiki/Q124149651
OpenAlex ID: keywords/pretraining
2018 · Devlin, Jacob, Chang, Ming-Wei, et al. · arXiv (Cornell University) · 77,072
2019 · Jacob Devlin, Ming-Wei Chang, et al. · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · 57,890
2020 · Dosovitskiy, Alexey, Beyer, Lucas, et al. · arXiv (Cornell University) · 51,442
2020 · Brown, Tom B., Mann, Benjamin, et al. · arXiv (Cornell University) · 33,796
2019 · Sanh, Victor, Debut, Lysandre, et al. · arXiv (Cornell University) · 10,350
2017 · Joao Carreira, Andrew Zisserman · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 9,983
2026 · Wayne Xin Zhao, Kun Zhou, et al. · Frontiers of Computer Science · 8,586
2022 · Salman Khan, Muzammal Naseer, et al. · ACM Computing Surveys · 4,064
2022 · Zhang, Hao, Li, Feng, et al. · arXiv (Cornell University) · 3,260
2011 · G. E. Dahl, Dong Yu, et al. · IEEE Transactions on Audio Speech and Language Processing · 3,052
2017 · Chen Sun, Abhinav Shrivastava, et al. · IEEE/CVF International Conference on Computer Vision (ICCV) · 2,652
2020 · Xiaoqi Jiao, Yichun Yin, et al. · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · 2,435
2021 · Tolstikhin, Ilya, Houlsby, Neil, et al. · arXiv (Cornell University) · 2,411
2009 · Ruslan Salakhutdinov, Geoffrey E. Hinton · International Conference on Artificial Intelligence and Statistics · 1,890
2018 · Agnieszka Mikolajczyk, Michal Grochowski · International Interdisciplinary PhD Workshop (IIPhDW) · 1,826
2022 · Tianyang Lin, Yuxin Wang, et al. · AI Open · 1,729
2019 · Wang, Alex, Pruksachatkun, Yada, et al. · arXiv (Cornell University) · 1,646
2022 · Xiaoyi Dong, Jianmin Bao, et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 1,552
2020 · Ilias Chalkidis, Manos Fergadiotis, et al. · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · 1,487
2019 · Simon Kornblith, Jonathon Shlens, et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 1,480
2019 · Hu, Weihua, Liu, Bowen, et al. · arXiv (Cornell University) · 1,427
2019 · Zhang, Jingqing, Zhao, Yao, et al. · arXiv (Cornell University) · 1,418
2022 · Xumin Yu, Lulu Tang, et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 1,405
2019 · Su, Weijie, Zhu, Xizhou, et al. · arXiv (Cornell University) · 1,395
Works count: 7,290
Citation count: 436,200
Year
Topic
Type