Toward Co-Working with Autonomous Agents: Rethinking Operations of Automation Systems
Year: 2025
Type: conference-paper
Abstract: The growing complexity of commissioning industrial automation systems has increased the demand for simpler and more intuitive operation, often requiring domain-specific expertise. This paper explores task-oriented natural-language interaction with automation systems, a paradigm that is rapidly gaining traction particularly due to the advent of large language models (LLMs). Leveraging a multi-agent system (MAS), the authors introduce a two-phase approach for controlling an industrial gantry for pick-and-place operations. In phase one, the MAS generates an execution plan based on the user input; in phase two, the plan is executed deterministically on real-world hardware. Our preliminary results demonstrate the advantages of a chat-based human-in-the-loop (HiL) workflow for refining and validating execution plans, and they highlight the promise of natural-language interfaces for operating pick-and-place systems (Demo: Video 1, Video 2).
Authors: Martin Kovacs, Johannes Baumgartl, Fabian Schreier, Ferdinand Hummel, Jan Seyler, Shahram Eivazi
Institutions: Festo (Germany), Esslingen University of Applied Sciences
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4415398120
FWCI:
Cites:
Cited by:
Related to: