Work

Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planning

Year: 2025

Type: conference-paper

Abstract: Large language models (LLMs) have revolutionized a large variety of NLP tasks. An active debate is to what extent they can do reasoning and planning. Prior work has assessed the latter in the specific context of PDDL planning, based on manually converting three PDDL domains into natural language (NL) prompts. Here we automate this conversion step, showing how to leverage an LLM to automatically generate NL prompts from PDDL input. Our automatically generated NL prompts result in similar LLM-planning performance as the previous manually generated ones. Beyond this, the automation enables us to run much larger experiments, providing for the first time a broad evaluation of LLM planning performance in PDDL. Our NL prompts yield better performance than PDDL prompts and simple template-based NL prompts. Compared to symbolic planners, LLM planning lags far behind; but in some domains, our best LLM configuration scales up further than A* using LM-cut.

Source: Proceedings of the International Conference on Automated Planning and Scheduling

DOI: 10.1609/icaps.v35i1.36126

Authors: Katharina Stein, Daniel Fišer, Jörg Hoffmann, Alexander Koller

Institutions: Saarland University, Aalborg University, German Research Centre for Artificial Intelligence

Language: English

Topic: AI-based Problem Solving and Planning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funder: Deutsche Forschungsgemeinschaft

Awards: G1951765764, GRK 2853: Neuroexplizite Modelle für Sprachverarbeitung, Bilderkennung und Aktionsentscheidungen, SFB 1102: Informationsdichte und sprachliche Kodierung