Domain generalization and punch mark classification
Year: 2025
Type: article
Abstract: • Convolutional neural networks show capacity to classify punch marks. • Classification often fails for paintings not in the training data. • Several domain generalization methods appear to offer little improvement. • Punch mark classification gives insight on domain generalization strategies. Punch tools were used to mechanically make decorative impressions—called punch marks—on gold ground paintings, becoming particularly widespread in Italy during the 14th and 15th centuries. Punch tools were frequently reused for multiple paintings, sometimes from different workshops, and the impressions they leave behind support a variety of art-historical investigations by evidencing workshop practices, attribution, contemporary connections, and more. In particular, classifying punch marks in paintings enables comparisons with extensive indices that were created by art historians/conservators Erling S. Skaug and Mojmír Frinta over the course of several decades. In this paper, we explore the potential of automated methods for punch mark classification. As in most image analysis tasks, deep neural networks are state of the art. Indeed, convolutional neural networks can produce highly accurate classification results, but often falter when confronted with images from paintings not represented in the training data. This is a particularly relevant problem in cultural heritage applications such as punch mark classification, where the size of training sets is typically small. For this reason, we have explored domain generalization methods, which aim to maximize accuracy on some target domain (images from a painting unseen in training data) using various source domains (images from paintings used for training). We find that, despite their promise, domain generalization methods (with explicit domain labels) unfortunately often offer little advantage over baseline convolutional neural networks (without explicit domain labels). Our results provide insight on the capacity and limitations of several off-the-shelf deep learning methods to automatically classify punch marks. Conversely, the structure and the challenges of punch mark images provide an interesting test case for insight on domain generalization methods.
Source: Journal of Cultural Heritage
Institution: University of Cambridge
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,380
Funders: Wellcome Trust, Dell EMC, Alan Turing Institute, UK Research and Innovation, National Institute for Health and Care Research, Leverhulme Trust, Department of Health and Social Care, Royal Society, European Commission, Engineering and Physical Sciences Research Council, Science and Technology Facilities Council, Horizon 2020, NIHR Cambridge Biomedical Research Centre
Awards: All in one cancer imaging optimisation using an integrated mathematical and deep learning approach, G1424764954, G1649034856, G1755984640, G2493601830, G2537968379, Robust and Efficient Analysis Approaches of Remote Imagery for Assessing Population and Forest Health in India, The Mathematics of Deep Learning, G3243167903, DTP 2224 University of Cambridge, G5275711260, G581313191, Cambridge Mathematics of Information in Healthcare (CMIH), G6153437118, EPSRC Centre for Mathematical and Statistical Analysis of Multimodal Clinical Imaging, Cambridge Service for Data Driven Discovery (CSD3) - A National Data Intensive Science Cloud for Converged Simulation, AI & Analytics, G7531310420, Combining Knowledge And Data Driven Approaches to Inverse Imaging Problems, PET++: Improving Localisation, Diagnosis and Quantification in Clinical and Medical PET Imaging with Randomised Optimisation, Predicting Dementia: Optimising and translating AI to improve prognosis and clinical pathways, G8355147377
OpenAlex ID: W4408072906