Work

Towards collective intelligence in agriculture: Deep reinforcement learning and digital twins for efficient management of collective irrigation water distribution systems

Year: 2025

Type: article

Abstract: • The paper developed a digital twin (DT) for optimizing Open-Channel Irrigation Systems (OCIS) evaluated in a semi-arid irrigated district in Morocco. • The study evaluated a Deep Deterministic Gradient Policy (DDPG) reinforcement learning agent and Genetic Algorithms (GA) for optimizing the spatiotemporal distribution of sowing dates. • The optimization problem is formulated as a constrained maximization of a Lagrangian function representing Water Use Efficiency (WUE), subject to the constraints imposed by the OCIS. • The agent successfully learned the dynamics of the digital twin (DT), as evidenced by increasing reward signals and decreasing loss curves for both the Actor and Critic Networks. • The agent-recommended sowing dates scenario demonstrated superior WUE, achieving a 4.69% increase in total crop yield compared to fixed-date sowing and the GA recommendation, while effectively adhering to the constraints imposed by the OCIS. Optimizing traditional Open-Channel Irrigation Systems (OCIS) is crucial for enhancing water sustainability and food security in water-scarce regions. OCIS remains widely used in many countries but receives less research attention in optimization studies compared to the rapid advancements in precision agriculture, which leverage new technologies such as artificial intelligence, remote sensing, drones, and the internet of things. Here, this study aims to optimize Water Use Efficiency (WUE) in the context of OCIS by proposing a framework for an adaptive spatiotemporal distribution of sowing dates. The framework consists of an intelligent agent interacting with a Digital Twin (DT). R3 district, an irrigated area within the Tensift basin in Morocco was used as a study site. Agrometeorological data from an automatic weather station installed in R3 district feeds the DT. This latter operates currently as a Digital Model with no bidirectional data integration, and comprises a module based on the AquaCrop model to simulate crop growth, development, and yield, and another module using graph theory to simulate the OCIS. The study evaluates genetic algorithms (GAs), an evolutionary optimization technique, versus a deep deterministic policy gradient (DDPG) agent for recommending the adaptive spatiotemporal distribution of sowing dates. The objective function is formulated as a constrained maximization of a Lagrangian function representing WUE, subject to the constraints imposed by the OCIS. The DDPG agent was trained for 1000 epochs, while GA ran for 100 generations. Results showed that the DDPG agent effectively learned the environment's dynamics through interactions with the R3 DT, as evidenced by increasing trend in the reward signals. However, GAs showed prolonged stagnation at multiple plateaus suggesting that GAs were unable to escape suboptimal solutions. In terms of training time, GAs take the longest. Additionally, comparing a same-day sowing date scenario to the adaptive sowing dates scenario recommended by the DDPG agent revealed a 4.69% increase in the total crop yield with WUE of 1.95 while adhering to the hydraulic constraints of OCIS. This study marks a first step toward utilizing intelligent agents in critical areas such as irrigation water management. Future work will focus on enhancing the R3 DT to increase the agent's robustness, enabling it to generate reliable recommendations for real-world applications.

Source: Energy Nexus

DOI: 10.1016/j.nexus.2025.100599