Year: 2021
Type: preprint
Abstract: Transformer architecture has emerged to be successful in a number of natural language processing tasks. However, its applications to medical vision remain largely unexplored. In this study, we present UTNet, a simple yet powerful hybrid Transformer architecture that integrates self-attention into a convolutional neural network for enhancing medical image segmentation. UTNet applies self-attention modules in both encoder and decoder for capturing long-range dependency at different scales with minimal overhead. To this end, we propose an efficient self-attention mechanism along with relative position encoding that reduces the complexity of self-attention operation significantly from $O(n^2)$ to approximate $O(n)$. A new self-attention decoder is also proposed to recover fine-grained details from the skipped connections in the encoder. Our approach addresses the dilemma that Transformer requires huge amounts of data to learn vision inductive bias. Our hybrid layer design allows the initialization of Transformer into convolutional networks without a need of pre-training. We have evaluated UTNet on the multi-label, multi-vendor cardiac magnetic resonance imaging cohort. UTNet demonstrates superior segmentation performance and robustness against the state-of-the-art approaches, holding the promise to generalize well on other medical image segmentations.
Source: arXiv (Cornell University)
Authors: Gao, Yunhe, Zhou, Mu, Metaxas, Dimitris
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: CHS: Medium: Data Driven Biomechanically Accurate Modeling of Human Gait on Unconstrained Terrain, AitF: Collaborative Research: Topological Algorithms for 3D/4D Cardiac Images: Understanding Complex and Dynamic Structures, S&AS: FND: COLLAB: Adaptable Vehicular Sensing and Control for Fleet-Oriented Systems in Smart Cities, Phase 1 IUCRC Rutgers-New Brunswick: Center for Accelerated Real Time Analytics (CARTA), Machine Learning and Deformable Model-based 4D Characterization of Cardiac Dyssynchrony from MRI, CHS: Medium: Collaborative Research: Scalable Integration of Data-Driven and Model-Based Methods for Large Vocabulary Sign Recognition and Search
OpenAlex ID: W3181946646