Hierarchical Planning with Deep Reinforcement Learning for 3D Navigation of Microrobots in Blood Vessels
Year: 2022
Type: article
Abstract: Designing intelligent microrobots that can autonomously navigate and perform instructed routines in blood vessels, a crowded environment with complexities including Brownian disturbance, concentrated cells, confinement, different flow patterns, and diverse vascular geometries, can offer enormous opportunities and challenges in biomedical applications. Herein, a biological‐agent mimicking a hierarchical control scheme that enables a microrobot to efficiently navigate and execute customizable routines in simplified blood vessel environments is reported. The control scheme consists of two decoupled components: a high‐level controller decomposing complex navigation tasks into short‐ranged, simpler subtasks and a low‐level deep reinforcement learning (DRL) controller responsible for maneuvering microrobots to accomplish subtasks. The proposed DRL controller utilizes 3D convolutional neural networks and is capable of learning control policies directly from raw 3D sensory data. It is shown that such a control scheme achieves effective and robust decision‐making within unseen, diverse complicated environments and offers flexibility for customizable task routines. This study provides a proof of principle for designing intelligent control systems for autonomous navigation in vascular networks for microrobots.
Source: Advanced Intelligent Systems
Authors: Yuguang Yang, Michael A. Bevan, Bo Li
Institutions: Johns Hopkins University, Tsinghua University
Language: English
Topic: Micro and Nano Robotics
Subfield: Condensed Matter Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,900
Awards: G1067211356, G4052616657, G8174409899
OpenAlex ID: W4297106277
