Work

ViT-HVE: a vision transformer-based framework for recognition and weighted evaluation of cultural heritage values

Year: 2025

Type: article

Abstract: Cultural heritage value serves as a critical foundation for guiding the protection and utilization of cultural heritage. However, existing methods for evaluating cultural heritage value face several inherent limitations, including evaluator subjectivity, high resource demands, and a lack of timely and objective evaluation capabilities. To overcome these challenges, we present the first application of deep learning to heritage value evaluation through our proposed Vision Transformer-based Heritage Value Evaluation model (ViT-HVE). This novel framework represents a breakthrough in automated cultural heritage analysis, enabling both visual recognition and quantitative evaluation of heritage values through an end-to-end deep learning approach. In this paper, the cultural heritage of the Yellow River in Shaanxi province is selected as a case study. A ten-dimensional value system characterizing these cultural heritage is extracted using the Latent Dirichlet Allocation (LDA) topic model. Based on this, we developed the first Cultural Heritage Value Recognition (CHVR) dataset in this domain. To facilitate the quantitative evaluation of cultural heritage values, we further introduce a Top-k Heritage Value Weighting (TK-HVW) method, which performs value weighting by extracting and normalizing the probability distributions over heritage value categories. To evaluate the effectiveness of the proposed method, we conduct a comparative analysis between ViT-HVE and five state-of-the-art deep learning models, and validate the accuracy of the model’s weighted outputs using expert evaluation. Furthermore, three distinct fine-tuning strategies are systematically investigated to enhance the performance of our model on the CHVR dataset. Experimental results indicate that ViT-HVE achieves the best performance in heritage value recognition tasks, and our proposed weighting method shows a high degree of consistency with expert evaluation results. The ViT-HVE evaluation method significantly enhances the objectivity and accuracy of cultural heritage recognition and evaluation, substantially reducing the cost and workload of related tasks, thereby offering scientific methodological support and practical guidance for the protection and utilization of cultural heritage.

Source: npj Heritage Science

DOI: 10.1038/s40494-025-02143-6