Multi-Objective Constrained Deep Q-Networks for Supplier Selection and Order Allocation in Sustainable Supply Chains
Year: 2026
Type: preprint
Abstract: Sustainable sourcing requires sequential decisions that couple discrete supplier admission, integer order quantities, stochastic lead times, inventory, and competing economic, environmental, and service objectives. We formulate this setting as a preference-conditioned constrained Markov decision process with state-dependent, mixed-integer action feasibility and distributional tail-risk requirements. We develop ParetoShield-Q, a multi-objective deep Q-learning formulation that uses an augmented Chebyshev action functional over distributional cost-to-go estimates and a completion-feasibility oracle to construct executable supplier allocations. The same constrained Bellman operator jointly determines which objectives are optimized, which downstream risks are penalized, and which supplier--quantity assignments are legally selectable. We establish conditional one-step feasibility and finite-candidate Pareto-efficiency properties, and provide an evaluation protocol based on the public M5 demand data and a public supplier-attribute dataset. The study presents seven quantitative tables and eight vector figures spanning multi-objective performance, preference control, cost composition, operational feasibility, ablations, disruption response, and computational scaling. The visualizations reproduce the tabulated numerical summaries without constructing unobserved run-level training traces. Independent reproduction requires the source training runs, scenario configurations, and optimizer logs.
Source: Research Square
Authors: Miao Li, Zihan Wang, Xuwen Cao, Yangcui Weng
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225566510