Get started
Work

An Attention Approach for Dimensionality Reduction That Can Correct Feature Collection Skewness

Year: 2022
Type: article
Abstract: It has been demonstrated that adding an attention mechanism to a convolutional neural network enhances network performance. However, in practice, the skewness error during the extraction process affects the feature information owing to the implementation of the global average pooling operation, which eventually lowers the performance of the network design. We think that when the weight is activated, the spatial information is effectively captured while the extraction range of the channel information is narrowed. We also think that because the weight is activated using multiple local extraction substitutions, the influence of skewness error can be effectively reduced. As a result, we suggest dimensionality reduction attention(RA) in this study as a new type of attention mechanism. Dimensionality reduction attention is a method of combining spatial information and channel information into a new feature distribution by aggregating dimensionality reduction of feature information, in contrast to other attentions that act on distinct feature tensors in space and channel. Under the premise of capturing long-distance context information and guaranteeing feature coordinates, it may realize the locality of spatial information through this operation and identify the local information of each channel. To completely display the important feature information, the produced feature maps are then encoded into two complimentary attention maps along the spatial and channel directions, respectively. Our dimension reduction focus is a straightforward and all-encompassing module. We run tests on various datasets using various deep architectures and various class designs. The outcomes of the trial demonstrate that our attention-based approach has clear benefits.
Source: IEEE Access
Authors: Ziyuan Wang, Li Wang
Language: English
Open Access status: gold
Estimated APC paid: $1,850
OpenAlex ID: W4312537529
FWCI: 0.162
Cites: 27
Cited by: 2
Related to: 10