Year: 2025
Type: article
Abstract: The present study looks at how integrating text, image, and video data through multi-modal learning could improve the abilities of Large Language Models (LLMs).The LLMs we have now been very good at processing natural words, but they could be even better if they could handle more than one type of input.A new framework that blends text-based LLMs, like GPT-4, with image and video models that use transformers and convolutional neural networks (CNNs) is what we're proposing.This method is used for jobs like visual question answering (VQA) and automated content generation, showing big gains in accuracy and understanding of the context.When compared to text-only models, our multi-modal model did 25% better on VQA standards.The system also improved the ability to create material by giving outputs that were richer and more context-aware.The results show that multi-modal learning can help LLMs make progress by helping them understand and react to different types of input better.
Author: Sreepal Reddy Bolla
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4413430368