Year: 2024
Type: article
Abstract: We propose a learning-based method to extract symbolic representations of the belief state and its dynamics in order to solve planning problems in a continuous-state partially observable Markov decision processes (POMDP) problem. While existing approaches typically parameterize the continuous-state POMDP into a finite-dimensional Markovian model, they are unable to preserve fidelity of the abstracted model. The first major contribution of this paper is we propose a Neural Network based method to learn the non-Markovian transition model based on the Mori-Zwanzig (M-Z) formalism. Different from existing work in applying M-Z formalism to autonomous time-invariant systems, our approach is the first work generalizing the M-Z formalism to robotics, by addressing the non-Markovian modeling of the belief dynamics that is dependent on historical observations and actions. The second major contribution is we theoretically show that modeling the non-Markovian memory effect in the abstracted belief dynamics improves the modeling accuracy, which is the key benefit of the proposed algorithm. Simulation experiment of a belief space planning problem is provided to validate the performance of the proposed belief abstraction algorithms.
Source: Autonomous Robots
Authors: Mengxue Hou, Tony X. Lin, Enlu Zhou, Fumin Zhang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,690
Awards: G1285204305, G1960259822, G2057323907, GCR: Biomolecular Systems Engineering - Unlocking the Potential of Biological Programming, G4156403175, Addressing Input Model Uncertainty in Stochastic Simulation: From Quantification to Optimization, G5972442391, G6098521345, S&AS: INT: COLLAB: Goal-driven Marine Autonomy with Application to Fisheries Science and Management, MRI: Development of an Underwater Mobile Testbed Using a Software-Defined Networking Architecture, G8876996369
OpenAlex ID: W4405759993