Exploring Multimodal Large Language Models for Medical Image Captioning
Year: 2024
Type: dissertation
Abstract: Image captioning involves using models that combine methods from Computer Vision(CV) and Natural Language Processing (NLP) to generate textual descriptions of images. Inthe biomedical field, this process is known as Diagnostic Captioning (DC), where modelsautomatically generate diagnostic text from one or more medical images. DC not onlydescribes the images, but also interprets them to provide a diagnosis, helping healthcareprofessionals gain an initial understanding of a patient’s medical conditions. To achievethis, researchers have formulated DC models that can interpret images and generatedescriptions based on specified instructions. This study aims to unlock new possibilities inthe biomedical domain by harnessing the power of Multi-modal Large Language Models(MLLMs) through techniques like task-specific fine-tuning and few-shot learning. It alsoexplores innovative methods to integrate alternative image captions with a language model(LM) that has fewer parameters.
Author: Samprovalaki, Marina
Institution: Athens University of Economics and Business
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6949414471
Cites:
Cited by:
Related to: