Year: 2025
Type: dissertation
Abstract: The field of Artificial Intelligence (AI) has seen significant advancements, particularly with the rise of Deep Learning (DL) and Deep Neural Networks (DNNs), which have demonstrated remarkable capabilities in complex tasks such as image recognition, natural language processing, and autonomous systems. However, as AI systems become more integrated into critical domains such as healthcare, robotics, and transportation, the reliability of these systems has emerged as a key concern. One crucial aspect of reliability is the ability of DNNs to provide accurate confidence/uncertainty estimates, which reflect the true likelihood associated with their outputs - a concept known as uncertainty calibration. This thesis addresses the challenges associated with uncertainty calibration of DNNs aiming to enhance the reliability of these models in practical applications. The research is motivated by the recognition that, despite the increased accuracy of modern DNNs, these models are often poorly calibrated. Miscalibration can lead to overconfident or underconfident predictions, which undermines the applicability of AI systems in real-world scenarios specially in safety-critical applications where trustworthiness is paramount. The main objectives of this work are threefold: (i) to develop novel algorithms that improve the reliability of DNN predictions through uncertainty calibration, (ii) to establish robust evaluation frameworks for assessing the calibration of DNNs, and (iii) to ensure the practical applicability of calibrated models across diverse and complex real-world scenarios. To achieve the objectives, this thesis introduces innovative contributions to the field; firstly, it proposes new ensemble-based techniques for improving DNN calibration including both model-centric and data-centric approaches. These techniques, such as Agreement-Driven Dynamic Ensemble (ADDE) and Augmented Dynamic Inference (ADI), show robust performance against diverse state-of-the-art baselines in multiple representative experiments. Second, it is developed a comprehensive framework for evaluating uncertainty calibration in object detection, a more complex task than standard image classification. The proposed framework introduces new metrics and theoretical foundations that address the unique challenges intrinsic to uncertainty calibration in object detection problems, providing deeper insights into the semantic and spatial aspects of uncertainty. Finally, besides extending the concept of uncertainty calibration to object detection problems, this work also expands the empirical analysis to complex real-world data (e.g., remote sensing images), scenarios under distribution-shifts and out-of-distribution (OOD) data. In summary, this thesis advances the field of DL by addressing the crucial challenge of uncertainty calibration, working towards more reliable DNNs whose prediction can be interpreted from a probabilistic perspective. By enhancing how these models estimate and communicate uncertainty, this thesis works to ensures that AI systems can be trusted in critical applications where accurate risk assessment is essential. This aspect is vital for the continued integration of AI into domains where safety and trustworthiness cannot be compromised, marking an important step toward more dependable and applicable AI technologies.
Author: Conde, Pedro Rafael de Matos
Language: Portuguese
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6992794075