Work

Who Should Look at This Image? Small Language Agents Choosing the Tools of Inspection Drones

Year: 2026
Type: article
Abstract: Inspection drones can run small vision models on board and send images to larger models in the cloud. The drone must choose models for each image, with limited time and a weak radio link. We show that a small language model can learn to make this choice for power-line inspection. Step by step, the model picks one of about 60 vision tools and reads its answer. At the end it writes a report with three parts: the component type, whether there is a defect, and the condition. We train this agent by supervised fine-tuning and reinforcement learning and compare its reports with the true labels. We report four results. First, supervised fine-tuning turns a model that writes no correct report into an agent with 88.1% fully correct reports. Second, the set of tools matters: when we replace one large general model with a small component classifier, the agent learns to use two small classifiers on the drone, and 95.9% of its reports are fully correct, in 0.09 seconds per image and with no upload. Third, reinforcement learning with a reward on the final report raises fully correct reports from 79.8% to 88.4% and the share of defects found from 68% to 83%. Fourth, reinforcement learning helps the agents that we tested to adapt: when the general model is removed at test time, an agent with 1.7 billion parameters uses a tool that it never saw in training, and 92.4% of its reports are fully correct, against 76.2% without reinforcement learning.
Institution: TU Wien
Language: English
Open Access status: green
OpenAlex ID: W7226885948
FWCI:
Cites:
Cited by:
Related to: 10