Year: 2025
Type: article
Abstract: The study introduces a novel evaluation system designed to measure the metacognitive abilities of embodied agents. The system incorporates multiple metrics—including task success rate, self-monitoring accuracy (measured by AUC), error detection speed, and confidence calibration error—to provide a comprehensive assessment of an agent’s internal monitoring and self-regulatory processes. Experiments were conducted in simulated environments (using Meta-World, etc.) and on a real robotic platform performing target grasping tasks. Two types of agents were compared: baseline agents relying solely on external feedback and agents enhanced with integrated metacognitive modules. The results demonstrate that agents with metacognitive capabilities consistently achieve higher performance, exhibit more precise self-monitoring, and respond more swiftly to unexpected events. This evaluation system serves as a robust tool for assessing metacognitive functions and offers promising implications for the development of more adaptable and reliable autonomous systems in dynamic environments, thus significantly enhancing overall system performance continuously.
Author: Yuanhang Ling
Institution: National Research Tomsk State University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: Tomsk State University
OpenAlex ID: W4409778356