A comprehensive review of advances in transformer, GAN, and attention mechanisms: Their role in multimodal learning and applications across NLP
Year: 2025
Type: article
Abstract: The emergence and subsequent development of deep learning, specifically transformer-based architectures, Generative Adversarial Networks (GANs), and attention mechanisms, have had revolutionary implications on Natural Language Processing (NLP) and multimodal learning. Transformer models are neural network architectures that change an input sequence into an output sequence. Transformer architectures like the Generative Pre-Training Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) leverage self-attention mechanisms to enable high-level contextual learning as well as long-range dependencies. GANs are a kind of AI algorithm that is designed to solve generative modeling problems. Different GANs, such as StyleGAN and BigCAN, study a collection of training data and learn the distribution probabilities used to generate such datasets. Attention mechanisms, acting as the unifying thread between Transformers and GANs in multimodal learning, optimize deep learning models to attend to the most relevant parts of the input data. This paper explores the synergy between these technologies, emphasizing their combined potential in multimodal learning frameworks. In addition, the paper analyzes recent advancements, key innovations, and practical implementations that leverage Transformers, GANs, and attention mechanisms to enhance natural language understanding and generation.
Authors: Khan, Md Fokrul Islam, Begum, Mst Halema, Rahman, Md Arifur, Limon, Golam Qibria, Azam, Md Ali, Masum, Abdul Kadar Muhammad
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7081659646
FWCI:
Cites:
Cited by:
Related to: