Year: 2024
Type: conference-paper
Abstract: This paper investigates the potential of using natural language descriptions as an alternative to direct image-based observations for learning policies in reinforcement learning.Due to the inherent challenges in managing imagebased observations, which include abundant information and irrelevant features, we propose a method that compresses images into a natural language form for state representation.This approach allows better interpretability and leverages the processing capabilities of largelanguage models.We conducted several experiments involving tasks that required imagebased observation.The results demonstrated that policies trained using natural language descriptions of images yield better generalization than those trained directly from images, emphasizing the potential of this approach in practical settings.
Authors: Md Masudur Rahman, Yexiang Xue
Institution: Purdue University West Lafayette
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4401043622
