Motivated by emerging energy-intensive applications in electric vehicle charging and cloud computing, my research has focused on large-scale stochastic deadline scheduling problems under random task arrivals and processing cost. The objective is to minimize the expected sum of stochastic processing cost (due to intermittent renewable generation and fluctuating energy prices) and delay penalty cost (resulting from failures to finish tasks before user-specified deadlines). The hardness in these problems stems from the unknown dynamics of system uncertainties, and the high dimensionality in both the system state and action spaces. This talk overviews our research results on the rigorous establishment of structural optimal policy characterizations under discrete/continuous action spaces and convex/discrete delay penalties. The established optimal policy characterizations do not rely on any probabilistic assumption on the evolution of system uncertainties, and therefore can be naturally integrated into data-driven deep reinforcement learning (DRL) approaches for action space dimensionality reduction without loss of optimality. Numerical results on real-world data show that the proposed approach outperforms state-of-the-art DRL and MPC (model predictive control) based approaches.

Papers related to the talk:
10.1109/TSG.2020.3028470
10.1109/JIOT.2021.3111414

Dr. Xu is an Assistant Professor in the Department of Mechanical and Automation Engineering (MAE) at The Chinese University of Hong Kong (CUHK). After receiving the Ph.D. degree from the Massachusetts Institute of Technology (MIT) in 2012, he worked at the California Institute of Technology as a CMI (Center for the Mathematics of Information) postdoctoral fellow in 2012-2013. He was an assistant professor at the Singapore University of Technology and Design (SUTD) in 2013-2017. His research interests lie in stochastic optimal control, reinforcement learning, power system optimization, and electricity market design. Dr. Xu has 1 U.S. patent application and was a recipient of the MIT-Shell Energy Fellowship.

For more information about the ESD Seminar, please email esd_invite@sutd.edu.sg