Keyword

attention collapse

Description: A failure mode in attention mechanisms where attention weights become uninformative or overly concentrated, reducing model performance or useful information flow.
OpenAlex ID: keywords/attention-collapse
2021 · Zhou, Daquan, Kang, Bingyi, et al. · arXiv (Cornell University) · 861
2022 · Wang, Peihao, Zheng, Wenqing, et al. · arXiv (Cornell University) · 259
2023 · Xian Lin, Zengqiang Yan, et al. · Lecture notes in computer science · 135
2023 · Park, Namuk, Kim, Wonjae, et al. · arXiv (Cornell University) · 126
2024 · Guangjin Liang, Dianguo Cao, et al. · IEEE Transactions on Neural Systems and Rehabilitation Engineering · 56
2025 · Xuankun Rong, Wenke Huang, et al. · neural information processing systems · 48
2024 · Yaning Zhang, Qiufu Li, et al. · Pattern Recognition · 40
2023 · Qingping Zheng, Ling Zheng, et al. · Knowledge-Based Systems · 28
2024 · Zhehao Wang, Xian Lin, et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 25
2025 · Mingming Yang, Junhui Du, et al. · Biomedical Signal Processing and Control · 20
2026 · Sanyal, Sunny, Shwartz-Ziv, Ravid, et al. · Transactions on machine learning research · 16
2026 · Yuval Ran-Milo · Proceedings of the conference - Association for Computational Linguistics. Meeting · 15
2024 · Jiaojiao Li, Haile Li, et al. · IEEE Transactions on Geoscience and Remote Sensing · 15
2025 · Jihwan Kim, Miso Lee, et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 12
2026 · Luo, Cheng, Cai, Zefan, et al. · arXiv (Cornell University) · 10
2025 · Xiao, Xi, Tsaris, Aristeidis, et al. · arXiv (Cornell University) · 10
2025 · Jihwan Kim, Miso Lee, et al. · Pattern Recognition · 9
2023 · Shi, Bowen, Zhang, Xiaopeng, et al. · International Conference on Learning Representations · 9
2025 · Li, Huan, Luo, Longjun, et al. · arXiv (Cornell University) · 8
2026 · Long, Yancheng, Yang, Yankai, et al. · arXiv (Cornell University) · 7
Works count: 105
Citation count: 1,924
Year
Topic
Type