Keyword

induction heads

Description: Attention heads in transformer language models that implement a copying-like pattern mechanism supporting sequence continuation and in-context learning.
Primary topic: Topic Modeling (44%)
OpenAlex ID: keywords/induction-heads
2022 · Olsson, Catherine, Elhage, Nelson, et al. · arXiv (Cornell University) · 1,893
2024 · Gloeckle, Fabian, Idrissi, Badr Youbi, et al. · arXiv (Cornell University) · 368
2022 · von Oswald, Johannes, Niklasson, Eyvind, et al. · arXiv (Cornell University) · 316
2023 · Goldowsky-Dill, Nicholas, MacLeod, Chris, et al. · arXiv (Cornell University) · 252
2022 · Hernandez, Danny, Brown, Tom, et al. · arXiv (Cornell University) · 245
2023 · Alberto Bietti, Vivien Cabannes, et al. · neural information processing systems · 196
2024 · Nichani, Eshaan, Damian, Alex, et al. · arXiv (Cornell University) · 124
2024 · Kissane, Connor, Krzyzanowski, Robert, et al. · arXiv (Cornell University) · 75
2024 · Siyu Chen, Heejune Sheen, et al. · neural information processing systems · 72
2025 · Joy Crosbie, Ekaterina Shutova · Findings of the Association for Computational Linguistics: NAACL · 63
2023 · Reddy, Gautam · International Conference on Learning Representations · 59
2024 · Singh, Aaditya K., Moskovitz, Ted, et al. · arXiv (Cornell University) · 57
2024 · Makkuva, Ashok Vardhan, Bondaschi, Marco, et al. · arXiv (Cornell University) · 53
2025 · Jiajun Song, Zhuoyan Xu, et al. · Proceedings of the National Academy of Sciences · 44
2025 · Yin, Kayo, Steinhardt, Jacob · arXiv (Cornell University) · 41
2024 · Nived Rajaraman, Marco Bondaschi, et al. · neural information processing systems · 37
2024 · Li Ji-An, Corey Zhou, et al. · neural information processing systems · 23
2025 · Makkuva, Ashok Vardhan, Bondaschi, Marco, et al. · International Conference on Learning Representations · 17
2024 · Wang, Mingze, Yu, Ruoxi, et al. · arXiv (Cornell University) · 17
2025 · Baker, Garrett, Wang, George, et al. · International Conference on Learning Representations · 15
2025 · Zhang, Yuanzhao, Gilpin, William · International Conference on Learning Representations · 14
2025 · Haolin Yang, Hakaze Cho, et al. · neural information processing systems · 14
2024 · Taylor, Jordan K. · arXiv (Cornell University) · 14
2026 · Chen, Jianhui, Luo, Yuzhang, et al. · arXiv (Cornell University) · 13
2025 · Okpekpe, Destiny, Orvieto, Antonio · arXiv (Cornell University) · 12
Works count: 98
Citation count: 4,189
Year
Topic
Type