Robust Forensic Analysis of Strongly Compressed Images
Year: 2021
Type: dissertation
Abstract: Digital images and videos have taken an outstanding role in many areas of everyday life, e.g., for documentation and communication of events. However, the availability of sophisticated software applications makes it straightforward to realistically manipulate digital footage. This can entail detrimental consequences. The goal of multimedia forensics is to provide as much information as possible on the origin, history and authenticity of multimedia samples. Over the past two decades, numerous successful algorithms have been proposed to address this goal. One of the major contemporary challenges of multimedia forensics is to maintain algorithm performance under strong lossy compression. Lossy compression sacrifices signal fidelity for reduced bit rates, and is particularly widespread in online and mobile applications. In this dissertation, we present several contributions for robust forensic analysis of strongly compressed images. First, we propose a taxonomy of existing multimedia forensics algorithms, that categorizes approaches based on their relation to compression. We identify three major groups: The family of statistics-based algorithms being impeded by compression, the family of compression-based algorithms that are symbiotic to compression, and the family of physics-based approaches that are largely insensitive to compression. Based on this categorization, we identify common strengths and major limitations, as well as potential remedies. Second, we make several algorithmic contributions in the group of physics-based and statistics-based methods for robust and flexible forensic image analysis. Arguably the main limitations of physics-based approaches are their restrictive assumptions on scene composition, and often requiring manual annotations. Different from other color-based works that solely model the illuminant color, we propose a more descriptive forensic cue that jointly models the influence of in-camera processing and illuminant conditions in a supervised fashion. We further propose a metric learning-based extension of the color descriptor, that requires much weaker supervision, and thereby is amenable to significantly larger training datasets, enhancing performance. We show that our proposed descriptor is very robust against compression, and that it outperforms state-of-the-art splicing detectors on low-quality images, without being restricted to particular scene composition and without requiring user input. One of the main limitations of statistics-based approaches is that they typically strongly deteriorate in the presence of compression. To assess their real-world applicability to compressed images, it is critical to evaluate algorithms on rigorous realistic test cases. We argue that this is infeasible for camera identification using existing databases, and propose a novel database to close this gap. Using this database, we investigate the robustness of learning-based camera identification. We present an approach that significantly outperforms the state-of-the-art in camera identification, both on clean and strongly compressed images. We further find that using compression as data augmentation can significantly improve performance on compressed images, even for completely unknown compression algorithms.
Author: Hadwiger, Benjamin
Institution: Friedrich-Alexander-Universität Erlangen-Nürnberg
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6996411997
