Year: 2019
Type: preprint
Abstract: The multi-armed bandit (MAB) model has been widely adopted for studying many practical optimization problems (network resource allocation, ad placement, crowdsourcing, etc.) with unknown parameters. The goal of the player here is to maximize the cumulative reward in the face of uncertainty. However, the basic MAB model neglects several important factors of the system in many real-world applications, where multiple arms can be simultaneously played and an arm could sometimes be "sleeping". Besides, ensuring fairness is also a key design concern in practice. To that end, we propose a new Combinatorial Sleeping MAB model with Fairness constraints, called CSMAB-F, aiming to address the aforementioned crucial modeling issues. The objective is now to maximize the reward while satisfying the fairness requirement of a minimum selection fraction for each individual arm. To tackle this new problem, we extend an online learning algorithm, UCB, to deal with a critical tradeoff between exploitation and exploration and employ the virtual queue technique to properly handle the fairness constraints. By carefully integrating these two techniques, we develop a new algorithm, called Learning with Fairness Guarantee (LFG), for the CSMAB-F problem. Further, we rigorously prove that not only LFG is feasibility-optimal, but it also has a time-average regret upper bounded by $\frac{N}{2η}+\frac{β_1\sqrt{mNT\log{T}}+β_2 N}{T}$, where N is the total number of arms, m is the maximum number of arms that can be simultaneously played, T is the time horizon, $β_1$ and $β_2$ are constants, and $η$ is a design parameter that we can tune. Finally, we perform extensive simulations to corroborate the effectiveness of the proposed algorithm. Interestingly, the simulation results reveal an important tradeoff between the regret and the speed of convergence to a point satisfying the fairness constraints.
Source: arXiv (Cornell University)
Authors: Li, Fengjiao, Liu, Jia, Ji, Bo
Institutions: Iowa State University, Temple University
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Awards: G2008031691, G2627375560, NeTS: Small: Toward Optimal, Efficient, and Holistic Networking Design for Massive-MIMO Wireless Networks, CIF: Small: Taming Convergence and Delay in Stochastic Network Optimization with Hessian Information, G4504108201, SpecEES: Toward Spectral and Energy Efficient Cross-Layer Designs for Millimeter-Wave-Based Massive MIMO Networks, CPS: Synergy: Collaborative Research: Cognitive Green Building: A Holistic Cyber-Physical Analytic Paradigm for Energy Sustainability, CAREER: Theory and Algorithms for Efficient Control of Wireless Networks with Jointly Optimized Performance: High Throughput, Low Delay, and Low Complexity, CRII: CIF: Models, Theories and Algorithms for Timeliness Optimization in Information-update Systems, G8876996369, G8929337198
OpenAlex ID: W2910142039