Work

Hierarkiat, haku ja generatiiviset mallit peräkkäisessä päätöksenteossa

Year: 2025
Type: dissertation
Abstract: Intelligent agents trained using deep reinforcement learning (DRL) or imitation learning (IL) can achieve or even surpass expert-level performance in various controlled settings. However, translating this success to real-world tasks remains difficult, especially in high-stakes domains such as cybersecurity, robotics, and autonomous driving (AD). Underlying issues include poor sample efficiency and brittle long-horizon reasoning. Basic IL frequently fails to capture complex distributions, such as human behavior. This thesis aims to tackle these limitations by developing algorithms that rely on hierarchical learning, search-based planning, and generative modeling, with discrete latent representations as a technical foundation. Hierarchical learning can shorten the planning horizon in extended tasks. Search-based planning can support reasoning and exploration. Generative models, particularly diffusion models, can represent multimodal distributions effectively in IL. Adversarial DRL, in turn, can be used to evaluate safety-critical systems. First, the thesis builds adversarial DRL agents to evaluate safety-critical systems. These agents are applied to operating system privilege escalation and pedestrian-collision generation to test leading AD algorithms. Second, it introduces a hierarchical IL algorithm that discovers data-driven hierarchies and solves long-horizon tasks via subgoal search. Third, it presents a search-based planner and sample-efficient world model learning for model predictive control (MPC). Fourth, the thesis empowers diffusion policies by conditioning them with driving styles learned from human carfollowing data using contrastive learning. A recurring theme throughout the thesis is the broad utility of discrete representations: capturing temporal abstraction in hierarchical learning, system state in MPC, and latent driving styles for generative modeling. Empirical evaluations across several domains suggest that combining hierarchy, search-based planning, and generative models can mitigate key obstacles in DRL and IL. Therefore, this thesis contributes tools and insights that could inform the design of more capable and reliable AI systems for practical problems in domains such as robotics, autonomous driving, and cybersecurity.
Institution: Aalto University
Language: English
Open Access status: green
OpenAlex ID: W7082411647
Cites: 163
Cited by:
Related to: 10