Unsupervised reward-driven image segmentation in automated scanning transmission electron microscopy experiments
Year: 2026
Type: article
Abstract: Automated experiments in scanning transmission electron microscopy (STEM) require rapid image segmentation to optimize the data representation for human interpretation, decision-making, site-selective spectroscopy, and atomic manipulation. Currently, segmentation tasks are typically performed using supervised machine learning methods, which require human-labeled data and are sensitive to out-of-distribution drift effects caused by changes in resolution, sampling, or beam shape. Here, we operationalize and benchmark a recently proposed reward-driven optimization workflow for on-the-fly image analysis in STEM. This unsupervised approach is much more robust because it does not rely on human labels and is fully explainable. The explanatory feedback can help humans verify the decision-making and potentially tune the model by selecting the position along the Pareto frontier of the reward functions. We established the timing and effectiveness of this method and demonstrated its capability for real-time performance in highthroughput and dynamic automated STEM experiments. The reward-driven approach allows for the construction of explainable robust analysis workflows and can be generalized to a broad range of image analysis tasks in electron and scanning probe microscopy and chemical imaging.
Source: Nanotechnology
Authors: Kamyar Barakati(Conceptualization, Data curation, Formal analysis, Methodology, Software, Writing – original draft), Utkarsh Pratiush(Investigation, Software, Writing – review & editing), Austin C Houston(Investigation, Software, Writing – review & editing), Gerd Duscher(Investigation, Supervision, Writing – review & editing), Sergei V Kalinin(Conceptualization, Formal analysis, Funding acquisition, Supervision, Writing – review & editing)
Institution: University of Tennessee at Knoxville
Language: English
Subfield: Structural Biology
Domain: Life Sciences
Open Access status: bronze
OpenAlex ID: W4403713484
