Safe Deep Learning in the Loop: Challenges, Methods, and Future Directions

Abstract:
Despite high-profile advances in various decision-making and classification tasks, Deep Neural Networks (DNNs) have found limited application in safety-critical domains such as self-driving cars and automated healthcare. In particular, DNNs can be vulnerable to adversarial attacks and input uncertainties. This issue becomes even more complicated when DNNs are used in closed-loop systems, where a small perturbation (caused by noisy measurements, uncertain initial conditions, disturbances, etc.) can substantially impact the system being controlled. Therefore, it is of utmost importance to develop tools that can provide useful certificates of stability, safety, and robustness for DNN-driven systems.

In this talk, I will present a new framework, inspired by robust control and based on semidefinite programming, for safety verification and robustness analysis of DNNs. The main idea is to abstract hard-to-analyze components of a DNN (e.g., the nonlinear activation functions) with the formalism of quadratic constraints. This abstraction allows us to reason about various properties of DNNs (safety, robustness, generalization, etc.) via semidefinite programming. I will conclude this talk by discussing future research directions as well as emerging challenges in realizing provably safe and robust learning-enabled systems.

Bio:
Mahyar Fazlyab received the Bachelor’s and Master’s degrees in mechanical engineering from Sharif University of Technology, Tehran, Iran, in 2010 and 2013, respectively. He earned a Master’s degree in statistics and a Ph.D. degree in electrical and systems engineering (ESE) from the University of Pennsylvania (UPenn), Philadelphia, PA, USA, in 2018. Currently, he is a Postdoctoral Researcher at UPenn. His research interests are at the intersection of optimization, control, and machine learning. His current work focuses on developing optimization-based methods for safety verification of learning-enabled control systems. Dr. Fazlyab won the Joseph and Rosaline Wolf Best Doctoral Dissertation Award in 2019, awarded by the ESE Department at UPenn.