Computer Vision and Historical Analysis of Scientific Illustration Circulation
Description: The VHS project proposes a new approach to the historical study of the circulation of scientific knowledge based on new methods of illustration analysis. Thanks to the recent developments in AI, and Computer Vision, the aim is to design a new analysis tool based on the study of the evolution and transformation of images in illustrated scientific corpora from the Middle Ages and the modern period. To this end, we will develop unsupervised or weakly supervised learning methods that will allow us to carry out large-scale automatic searches adapted to these corpora, based on the detection of iconographic similarities (between images, in order to identify the copying and borrowing processes, in particular) and textual similarities (between images and captions or associated texts, in order to identify, e.g., different images describing similar textual content). These methods will provide historians with new associations of illustrations and possible relationships (inter-iconic and/or inter-textual), the analysis of which will allow to answer to several essential questions in studies of the circulation of illustrated scientific knowledge, starting with the place and role of the image in the transmission of these knowledge. To achieve these goals, the project involves three recognized partners (the Digital Humanities team of the Institut des Sciences du Calcul et des Données at Sorbonne University; the Monde Byzantin team from the Orient & Méditerranée laboratory (UMR 8167); the Imagine team from the Gaspard Monge Computer Science Laboratory at École des Ponts ParisTech (ENPC)) which brings together specialists in History of Science, scientific illustration, Computer Vision and Deep Learning. The VHS team is driven by a strongly interdisciplinary approach that will be structured over the four years of the project around regular internal workshops, which will allow the cross-fertilization of skills and approaches and the organization of work, and a monthly public seminar dedicated to studies on scientific illustration circulation. The work will be carried out on four illustrated corpora, two medieval manuscript corpora and two printed corpora from the modern period, covering three fields of knowledge: natural history, mathematics and pharmacology. These corpora will be indexed, and their illustrations automatically extracted in an IIIF database equipped with a consultation and annotation interface, the implementation of which will constitute the first phase of the project. The consortium will then work in parallel on the development of similarity detection methods, the joint analysis of the tests carried out, as well as the historical study of the results obtained and the associated methodological issues. The approach adopted, closely combining humanities and data sciences, places the project firmly in the field of Digital Humanities, where VHS will provide a concrete and important result for historians of science, of technology, of art and for the visual studies community: the availability of the methods developed in the form of an IIIF API, designed as a research environment for the analysis of the circulation of scientific knowledge through images. These same methods will also constitute original contributions in the field of AI, on several important challenges, such as weakly supervised learning of fine-grained image representations (including weakly aligned text data) and weakly supervised learning of image style. The results of the project will be disseminated academically (publications, conferences, preparation of a collective work) in all the disciplines involved. They will also be pedagogically valorized through the development of educational materials for primary and secondary school teachers, as well as the implementation of a Master's degree Teaching Unit in Digital Humanities.
Funder: Agence Nationale de la Recherche
Amount: €612,400
Funding type: research
Funder scheme: AAPG2021
Start date: 2022-01-01
Investigator: Stavros LAZARIS
Institution: Orient et Méditerranée, textes - archéologie - histoire
Funded outputs count: 7
Funder award ID: ANR-21-CE38-0008
OpenAlex ID: G2425706135