Work

Insights into LLM Long-Context Failures: When Transformers Know but Don’t Tell

Year: 2024

Type: conference-paper

Abstract: Large Language Models (LLMs) exhibit positional bias, struggling to utilize information from the middle or end of long contexts.Our study explores LLMs' long-context reasoning by probing their hidden representations.We find that while LLMs encode the position of target information, they often fail to leverage this in generating accurate responses.This reveals a disconnect between information retrieval and its communication, a 'know but don't tell' phenomenon.We further analyze task accuracy vs layers, offering insights into the underlying mechanics of transformer models.* ✰ indicates equal contributions.* The code for reproducing our results is accessible here.

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/v1/2024.findings-emnlp.447

Authors: Muhan Gao, TaiMing Lu, Kuai Yu, Adam Byerly, Daniel Khashabi

Institution: Johns Hopkins University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Allen Institute for Artificial Intelligence, Office of Naval Research

Award: G8876996369