DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures
Year: 2026
Type: conference-paper
Abstract: We propose a novel self-attention mechanism for document understanding that takes into account text block positions in relative polar coordinate system rather than the Cartesian one.Based on this mechanism, we build DOCPO-LARBERT, a layout-aware BERT model for document understanding that eliminates the need for absolute 2D positional embeddings.Despite being pre-trained on a dataset more than six times smaller than the widely used IIT-CDIP corpus, DOCPOLARBERT achieves state-of-the-art results.These results demonstrate that a carefully designed attention mechanism can compensate for reduced pre-training data, offering an efficient and effective alternative for document understanding.
Institutions: Centre National de la Recherche Scientifique, Laboratoire de Mathématiques de Bretagne Atlantique, Institut national de recherche en sciences et technologies du numérique, Institut de Recherche en Informatique et Systèmes Aléatoires, Institut National des Sciences Appliquées de Rennes, Université de Rennes
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140107208