Seeing is Believing: Certified Perception-Based Control from Learned Visual Representations via System Level Synthesis
Year: 2026
Type: conference-paper
Abstract: We propose VISION-SLS, a method for nonlinear output-feedback control from high-resolution RGB images which provides robust constraint satisfaction guarantees under calibrated uncertainty bounds despite partial observability, sensor noise, and nonlinear dynamics.To enable scalability while retaining guarantees, we propose: (i) a learned lowdimensional observation map from pretrained visual features with state-dependent error bounds, and (ii) a causal affine time-varying output-feedback policy optimized via System Level Synthesis (SLS).We develop a scalable, novel solver for the resulting nonconvex program that leverages sequential convex programming coupled with efficient Riccati recursions.On two simulated visuomotor tasks (a 4D car and a 10D quadrotor) with 512 ⇥ 512 pixels and a 59D humanoid task with partial observability, our method enables safe, information-gathering behavior that reduces uncertainty while guaranteeing constraint satisfaction with empirically-calibrated error bounds.We also validate our method on hardware, safely controlling a ground vehicle from onboard images, outperforming baselines in safety rate and solve times.Together, these results show that learned visual abstractions coupled with an efficient solver make SLS-based safe visuomotor output-feedback practical at scale.
Source: Robotics: Science and Systems
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: bronze
Awards: G7301659660, G8270931710
OpenAlex ID: W7204817606
