Year: 2025
Type: conference-paper
Abstract: In this work, we present a method for learning 2D triangular meshes from images using the Transformer architecture.Creating polygonal representations of 2D surfaces from images has a wide range of applications, e.g.texture mapping and image segmentation.Current machine learning based methods either rely on deforming template meshes with fixed topology or require expensive post-processing to extract a planar polygonal mesh.In this paper, we demonstrate that deep learning can be utilized to directly generate a planar polygonal surface from arbitrary images without the need for additional input or constraints.Specifically, we show that the attention mechanism in transformer networks is highly effective in learning a unified representation of vertex positions and their neighborhood relationships.To showcase this, we propose a self-supervised training pipeline that enables end-to-end learning of meshes directly from images.We apply our approach to various datasets of handwritten letters and digits, and show that the model is capable of learning meshes with varying genus.
Source: Computer Science Research Notes
Institutions: Graz University of Technology, Fraunhofer Austria
Language: English
Subfield: Computational Mechanics
Field: Engineering
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4412763429