Generative Intelligence for Robotic Manipulation: A Framework for Proactive Planning, Failure Diagnosis, and Autonomous Recovery
Year: 2025
Type: dissertation
Abstract: The deployment of autonomous robots in complex, unstructured environments remains a significant challenge.This difficulty is fundamentally limited by the weakness of contemporary control and planning systems.Classical approaches to task and motion planning are powerful in constrained domains.However, they often fail in practice, especially when faced with unforeseen perturbations or model inaccuracies.Recent advances in generative intelligence, such as Large Language Models (LLMs), Vision Language Models (VLMs), and diffusion-based models, have provided a new foundation for robot learning.While these approaches show promise, they operate in isolation.Some systems can detect anomalies, while others can produce human-readable explanations or learn recovery policies from large volumes of failure data.Yet, they do not form a unified, autonomous loop that links failure detection to causal diagnosis and targeted recovery.This gap also constrains their practical applicability in real-world settings.This thesis argues that achieving true autonomy requires a paradigm shift from purely reactive error correction to holistic, proactive intelligence that anticipates, diagnoses, and recovers from failures.We introduce a novel framework: Generative Intelligence for Robotic Manipulation.This framework unifies four critical capabilities within a single closed-loop architecture.First, a proactive planning module uses an LLM to decompose high-level tasks into verifiable, constrained action steps.Second, a VLM monitors execution in real-time to detect when these constraints are violated.Third, upon failure, the VLM performs a deeper, causal diagnosis to generate a structured, natural-language explanation of why the failure occurred.Finally, this explanation is fed back to the LLM planner, which formulates an intelligent recovery plan that directly addresses the diagnosed cause.Furthermore, successful failure-recovery cycles are summarized into reusable lessons and stored in an Adaptive Knowledge Base.This enables the system to learn from its experiences and avoid predictable errors that may occur in the future.
Author: Phung, Quang Huy
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7215671029