Work

Advances in multimodal deep learning for skin lesion recognition : attention, architecture optimization, and robust clinical integration

Year: 2026
Type: dissertation
Abstract: Automatic skin lesion classification has advanced substantially with deep learning in recent years, yet many systems still rely predominantly on image information while making limited use of complementary clinical data. This restricts their ability to more realistically reproduce the dermatological diagnostic process, which naturally combines visual evidence with patient context. In this scenario, multimodal strategies for skin lesion classification were investigated, with emphasis on the integration of dermoscopic images and clinical metadata. The problem was addressed from four complementary perspectives: explicit modeling of image-metadata interaction through attention-based mechanisms; analysis of the influence of clinical attributes on the decision-making process; development of more stable and controlled multimodal fusion strategies; and automatic search for competitive architectures under practical constraints. In addition, robustness to incomplete metadata was explored through semantically richer textual representations as an alternative to traditional categorical encodings. The results indicate that carefully designed multimodal architectures can outperform unimodal approaches and simple fusion strategies, particularly in terms of balanced accuracy. They also show that the way metadata is incorporated directly affects model stability, interpretability, and robustness, while residual and attention-based mechanisms contribute to a more controlled fusion process. In parallel, architecture optimization through search demonstrated potential to identify compact and competitive solutions. Taken together, the Ąndings contribute to the advancement of multimodal skin lesion analysis by improving the understanding of the role of clinical attributes, proposing more adaptive fusion strategies, and exploring solutions oriented toward robustness and efficiency in realistic clinical scenarios
Language: Portuguese
Subfield: Oncology
Field: Medicine
Open Access status: green
OpenAlex ID: W7221841794
Cites:
Cited by:
Related to: 9