The optimal operation of water resources systems is a wide and challenging application domain for optimal control methods and tools. Water resources systems are highly non linear systems, with large dimensional state and control spaces, affected by uncertainty and characterized by multiple competing objectives.  Most of the operational problems involving these systems can be formulated as Markov decision processes and solved via Dynamic Programming (DP) or Reinforcement Learning. Although DP family methods can be applied under mild assumptions, they suffer from three well known curses, considerably constraining their wide application: i) the curse of dimensionality, namely the computational cost of DP grows exponentially with state and control space; ii) the curse of modeling, meaning the compulsory use of in-line model-based computations that make impossible the direct, model-free inclusion of exogenous information into the controller and/ or coupling DP with process-based simulation models; iii) the curse of multiple objectives, i.e. the computational effort grows factorially with the number of objectives considered. Direct Policy Search (DPS) methods have recently emerged as a flexible tool to overcome DP’s curses. DPS is a simulation-based optimization approach, where the optimal operating policy is searched for in a given class of parameterized functions, possibly non-linear approximating networks, and the control problem transformed into a more traditional optimization problem in the policy parameters’ space. In this talk, I will review the most recent advances in Evolutionary Multi-Objective Direct Policy Search (EMODPS), by presenting a diagnostic framework for the design and the evaluation of multipurpose reservoir operations, the development of efficient EMODPS parallelization schemes, and the applications of EMODPS to a range of real world problems, including robust design under global change and rival problem framings.


***
This seminar has been jointly organized by FEEM and IEFE, Bocconi University.