Reinforcement Learning-Based Navigation of Micro Robots in a Model of a Left Ventricle in the Presence of Aortic Regurgitation
Year: 2026
Type: dissertation
Abstract: Autonomous micro-robotic navigation within unsteady biological fluid environments represents one of the most challenging problems at the intersection of fluid mechanics, reinforcement learning, and biomedical engineering. This thesis investigates energy-efficient point-to-point navigation of micro-robots in the time-varying flow field of a model of a human left ventricle (LV) in the presence of a leaking aortic valve, with direct application to targeted intracardiac drug delivery. The central hypothesis is that a model-free reinforcement learning agent, trained without any explicit knowledge of the underlying fluid dynamics, can discover navigation policies that exploit the passive advective structures of the background flow while minimizing propulsive effort required to complete the navigation task. The proposed framework utilizes Proximal Policy Optimization (PPO) with a six-dimensional observation space comprising the agent's absolute coordinates, its relative displacement to the target, and the local flow velocity components. The reward structure is designed to incentivize flow exploitation over brute-force propulsion. The framework is first validated on the classical double-gyre benchmark flow and subsequently applied to a data-driven LV flow environment reconstructed from an in vitro cardiac velocity dataset. On the double-gyre benchmark, the PPO agent achieves a task success rate exceeding 90% while autonomously learning a passive drift strategy — deactivating its propulsion during favorable vortex phases to minimize energy expenditure. When applied to the LV flow environment, deterministic evaluation over 50 rollouts per condition yields 100% task success under both mild and moderate flow conditions, and 94% under the severe condition, with varying levels of energy consumption across regimes. These results provide a solid quantitative basis for the development of future autonomous micro-robotic drug delivery systems, advancing the prospect of localized therapeutic interventions for patients with cardiac disease. Keywords: Reinforcement Learning, PPO, Passive Advection, Energy Efficiency, Left Ventricular Flow
Author: Maleki, Hanieh
Institution: Concordia University
Language: English
Topic: Micro and Nano Robotics
Subfield: Condensed Matter Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221489446
