Keyword

attention head pruning

Description: A model-compression technique that removes selected attention heads from transformer networks to reduce computational cost or model size.
OpenAlex ID: keywords/attention-head-pruning
2019 · Elena Voita, David Talbot, et al. · Proceedings of the conference - Association for Computational Linguistics. Meeting · 2,123
2019 · Michel, Paul, Levy, Omer, et al. · arXiv (Cornell University) · 1,424
2021 · Yaru Hao, Li Dong, et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 347
2020 · Cordonnier, Jean-Baptiste, Loukas, Andreas, et al. · arXiv (Cornell University) · 312
2022 · Woosuk Kwon, Sehoon Kim, et al. · neural information processing systems · 267
2022 · Huy-Tan Thai, Kim-Hung Le, et al. · Computers and Electronics in Agriculture · 194
2021 · Han He, Jinho D. Choi · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 171
2024 · Zhuoran Jin, Pengfei Cao, et al. · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · 125
2022 · Pengyu Zeng, Guoliang Hu, et al. · Knowledge-Based Systems · 86
2023 · Yiran Li, Junpeng Wang, et al. · IEEE Transactions on Visualization and Computer Graphics · 79
2020 · Maximiliana Behnke, Kenneth Heafield · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 75
2025 · Guo, Jialong, Chen, Xinghao, et al. · arXiv (Cornell University) · 44
2019 · Baan, Joris, ter Hoeve, Maartje, et al. · arXiv (Cornell University) · 36
2021 · Zhengyan Zhang, Fanchao Qi, et al. · AI Open · 34
2022 · Przemyslaw Joniak, Akiko Aizawa · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · 32
2024 · Zhang, Zeliang, Pham, Phu, et al. · arXiv (Cornell University) · 31
2022 · Perez, Ilan, Reinauer, Raphael · arXiv (Cornell University) · 31
2023 · Haoyu Wang, Siyuan Wang, et al. · Interspeech (USB) · 29
2022 · Zejiang Hou, Sun-Yuan Kung · IEEE International Conference on Multimedia and Expo (ICME) · 28
2021 · Kyuhong Shim, Iksoo Choi, et al. · International SoC Design Conference · 28
2026 · Ji, Yicheng, Zhong, Zhizhou, et al. · arXiv (Cornell University) · 27
2022 · Laufer-Goldshtein, Bracha, Fisch, Adam, et al. · arXiv (Cornell University) · 27
2022 · Zejiang Hou, Sun-Yuan Kung · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · 27
2022 · Jiaoda Li, Ryan Cotterell, et al. · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · 27
2023 · William Held, Diyi Yang · Conference of the European Chapter of the Association for Computational Linguistics · 23
Works count: 192
Citation count: 6,066
Year
Topic
Type