Year: 2026
Type: article
Abstract: The digitization and online availability of herbarium specimens have increased the need for automated tools to extract specimen traits at scale. Quantitative trait data, such as leaf area or petal length, are critical for ecology, evolution, and climate change research. In recent years, automated methods such as LeafMachine2 (Weaver and Smith 2023) and Mothra (Wilson et al. 2022) have been developed to measure these traits from images, supporting large-scale studies. However, such image measurements need to be converted from pixels to standardized units (e.g., centimeters) to be useful. Existing methods estimate this conversion factor (CF) by detecting the ruler type and the tick mark locations directly. From these image coordinates, the average pixel distance between them can be calculated to estimate the CF. These methods typically require many annotated samples for each ruler type and tend to perform poorly when applied to unseen or visually distinct rulers. To overcome these limitations, we present a generic pipeline that accurately predicts the CF with few labeled rulers. The approach leverages the fact that many institutions reuse identical rulers, often differing only in institutional logos. Instead of directly classifying or predicting the tick mark locations, the pipeline uses a similarity-based approach. First, rulers on the page are detected using a YOLOv11 object detection model. Second, these rulers are cropped and normalized, ensuring the longest side is horizontal. After detection, we use Contrastive Language-Image Pre-training (CLIP) to embed each ruler and to retrieve the three most similar ruler candidates from our ground-truth dataset. Next, we use the LightGlue keypoint-matching model to detect corresponding keypoints between the query ruler and each retrieved candidate. This model detects and scores similar keypoint pairs (pixel coordinates) between two input image pairs. The candidate ruler with the highest average score across the 100 best keypoint pairs is selected as the best match. If none of the candidates had at least 10 matching keypoint pairs, we consider the query ruler a “no-match”. Finally, these keypoint pairs are used to estimate a homography, which describes how one image can be transformed to align with another taken from a different viewpoint. This transformation is then applied to the ground truth tick mark locations to calculate the CF for the query ruler. To evaluate the proposed method, we constructed a new dataset by randomly selecting 400 herbarium sheets from the Global Biodiversity Information Facility (GBIF), PlantCLEF2020, and Dillen et al. 2019 . Each ruler was annotated with its bounding polygon and a type. Ruler tick marks were annotated with their coordinate locations and units (e.g., ticks_1cm). The dataset contains a diverse set of 476 rulers from over 40 institutions. We estimated our measurement error at 0.794%, which is in line with the reported 0.8% from LeafMachine2, by calculating the average standard deviation of pixel distances between each tick mark. We evaluated our approach using a leave-one-out cross-validation, treating each ruler crop as a query and the remaining dataset as candidate matches. We also evaluated our method on the 1131 rulers from LeafMachine2 , which were annotated with a bounding box around the tick marks (see Fig. 1). Rulers without a matching type were excluded for evaluation. The method achieved an average CF error of 0.689% using the best-matching ruler. 94.8% of rulers had a CF error less than 2%. On the LeafMachine2 dataset, performance improved with an average CF error of 0.324%. There, 99.0% of rulers had a CF error less than 2%. We plan to investigate how different annotation strategies affect performance. Table 1 details these evaluation results. The proposed method offers several advantages over existing approaches. First, it allows an accurate conversion with fewer annotated rulers of the same type, essentially needing only one. Second, the keypoint matching process is robust to partial detections, occlusions, and low-resolution rulers, as visualized in Fig. 2. Third, only the detection model needs minimal fine-tuning when new ruler types are added to the dataset, as the matching models remain unchanged. Additionally, the prediction can still be accurate when the best-matching ruler is from a different institution (see Fig. 3). The main limitation of our method is that it requires at least one annotated ruler of a similar type to result in a good match. It also assumes the tick marks are on a straight line, resulting in poorer performance for curved rulers. Finally, we found that simple rulers, such as straight lines, tend to perform worse (see Fig. 3). This work represents part of our ongoing research to assist trait extraction at scale. We plan to expand the dataset with additional ruler types, including examples from other natural history collections. We will further refine the method to improve its robustness and generalization across institutions and to explore fallback methods when no match is found. The code and dataset are available on GitHub*1.
Institutions: Ghent University, Meise Botanic Garden
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7140086013