MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning
Year: 2025
Type: conference-paper
Abstract: In visual Reinforcement Learning (RL), learning from pixel-based observations poses significant challenges on sample efficiency, primarily due to the complexity of extracting informative state representations from high-dimensional data. Previous methods such as contrastive-based approaches have made strides in improving sample efficiency but fall short in modeling the nuanced evolution of states. To address this, we introduce MOOSS, a novelframe-work that leverages a temporal contrastive objective with the help of graph-based spatial-temporal masking to explicitly model state evolution in visual RL. Specifically, we propose a self-supervised dual-component strategy that integrates (1) a graph construction of pixel-based observations for spatial-temporal masking, coupled with (2) a multilevel contrastive learning mechanism that enriches state representations by emphasizing temporal continuity and change of states. MOOSS advances the understanding of state dynamics by disrupting and learning from spatial-temporal correlations, which facilitates policy learning. Our comprehensive evaluation on multiple continuous and discrete control benchmarks shows that MOOSS outperforms previous state-of-the-art visual RL methods in terms of sample efficiency, demonstrating the effectiveness of our method.
Institutions: University of Illinois Urbana-Champaign, Visa (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1146442242, G1353381076, G1453181110, G1503004319, Track 4: Advanced CI Coordination Ecosystem: Monitoring and Measurement Services, G1957999595, G2926720356, G298330452, G3562386289, Track 1: ACCESS Resource Allocations Marketplace and Platform Services (RAMPS), G4010157413, G4233114941, G4403015908, ACO: An Open CI Ecosystem to Advance Scientific Discovery (OpenCI), Track 3: COre National Ecosystem for CyberinfrasTructure (CONECT), G5218272324, G5367127336, G5464919783, G5620962805, CPS: Frontier: Collaborative Research: COALESCE: COntext Aware LEarning for Sustainable CybEr-Agricultural Systems, G8108599699, Track 2: Customized Multi-tier Assistance, Training, and Computational Help (MATCH) for End User ACCESS to CI, G8518781831, G8940766960
OpenAlex ID: W4409263154