Year: 2020
Type: dissertation
Abstract: Reinforcement learning is a field within machine learning which trains agents to solve problems by trial and error. A reinforcement learning agent interacts with the environment through actions. Depending on the outcome of these actions, the agent can either be rewarded or penalized. The overall goal of the autonomous agent is to maximize its objective function, which means obtaining the maximum amount of rewards. While the agent follows its objective function, its action can cause unexpected harm or unwanted changes to the environment and thus breach the safety. These unwanted events that might occur are defined as negative side effects. The focus in this work is to analyze approaches to avoid negative side effects and find solutions for this problem, without integrating ad-hoc constraints in the agent's objective function. These constraints are not integrated since we want the agent to have a general knowledge of solving side effects problems, as it is impossible to foresee all actions that could cause unexpected harm or unwanted changes in a real world environment. The experimental part compares two environments implemented in different settings, one in a safety setting and one in a standard setting. The agent in the safety setting is not directly aware of the penalization for causing side effects, as it is not a part of its objective. On the contrary, the agent in the standard setting is directly penalized and knows the consequence of causing harm. The experimental part shows that the agent in the standard setting performs well and is close to learning the optimal behaviour. Meanwhile, the agent in the safety setting does not behave as desired and fails to solve the problem of side effects.
Author: Waisi, Serwa
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7064023523