A Self-Learning Intelligent Control Framework for Networked Cyber-Physical Systems

About

The goal of this project is to address challenges in machine learning for intelligent physical systems that interact with one another. The approach is to explore Reinforcement Learning (RL) strategies, where systems are rewarded when behaving correctly, for interacting physical systems when the systems with which they interact may react in inconsistent ways. The results are expected to contribute to a new self-learning intelligent control framework, where the systems under design can decide how to interact with their inconsistent neighbors in a way that will improve how they learn. This will advance reinforcement learning for networked cyber-physical systems (CPS) which can have emergent behaviors when they interact (for example, unmanned aerial systems) and are frequently inconsistent due to uncertainties in their distributed nature (for example, the smart grid).

Three major fundamental contributions to the scientific field are expected. First, a new distributed RL algorithm will learn suitable reward functions automatically without requiring external supervisions. This work will relax human efforts and scale RL algorithms to more complex environment. Second, novel transfer learning-based RL architectures will be designed by reusing past knowledge from multiple sources. This design will further accelerate learning process in networked CPS. Third, this proposed method will be implemented on a multi-robot testbed to advance the learning in robot applications. Outreach and dissemination plans cultivate the scientific curiosity of K-12 students, and students from underrepresented groups, and motivate their interests in Science, Technology, Engineering, and Math (STEM) programs. Furthermore, the integration of the project's cutting-edge research results into new courses will aid retention of current STEM students.

 

Publications

Xie, Dong and Zhong, Xiangnan. "Deep Deterministic Policy Gradients with Transfer Learning Framework in StarCraft Micromanagement,"  2019 IEEE International Conference on Electro Information Technology (EIT) , 2019.