Work

Uncertainty Estimation in Deep Neural Networks: A Comparative Study of Bayesian Approximation and Conformal Prediction

Year: 2025
Type: dissertation
Abstract: Deep neural networks have been increasingly used in various scientific fields due to their versatility and high performance. Despite achieving high classification accuracy, deep learning models can be poorly calibrated, assigning overly confident probabilities to incorrect predictions. This overconfidence highlights the absence of built-in mechanisms for uncertainty quantification. The thesis compares two distinct approaches to uncertainty estimation: Bayesian approximation using Monte Carlo Dropout and the nonparametric Conformal Prediction framework. These methods are applied to predictions generated by two convolutional neural network architectures, H-CNN VGG16, and GoogLeNet, trained on the image classification benchmark dataset Fashion-MNIST. The results highlight that H-CNN VGG16 achieves higher accuracy but tends to be overconfident, while GoogLeNet results in better calibrated uncertainty estimation. Moreover, Conformal Predictions provide valid prediction sets that can compensate for unreliable or misleading uncertainty estimations. As a result, it is a valuable approach in high-risk settings. The thesis contributes to evaluating uncertainty estimation techniques by comparing Conformal Predictions to Bayesian approximation over two different architectures. It provides a deeper understanding of the need to assess a model’s performance beyond its accuracy.
Language: English
Open Access status: green
OpenAlex ID: W7081697980
Cites: 41
Cited by: 3
Related to: 10