Work

Attention routing in vision-language encoders: technical note

Year: 2026
Type: article
Abstract: Technical note recording the method and results, as of 14 November 2026, of a study of attention routing in the image encoders of pretrained vision-language models: the attention pattern of these encoders can be decided from a coarse (downsampled or fixed-size thumbnail) view of the image while content is read at full resolution; each query needs a small, fixed budget of keys chosen by the coarse view; and a coarse-routed block-sparse encoder scales near-linearly with resolution (3.9-4.3x faster than the unmodified encoder at about 50k patches on an A100) at matched accuracy on DocVQA, InfoVQA, TextVQA and ChartQA. SHA-256 of the deposited PDF (technical_note.pdf): da07771593364454c4d652fbbb76a213dbacab1876d13975ba2ab0f998985e1c
Language: English
Open Access status: green
OpenAlex ID: W7229214971
FWCI:
Cites:
Cited by:
Related to: 10