MechE Colloquium: Likelihood-weighted active learning with application to Bayesian optimization, uncertainty quantification, and decision making in high dimensions

Abstract

Analysis of physical and engineering systems is characterized by unique computational challenges associated with high dimensionality of parameter spaces, large cost of simulations or experiments, as well as existence of uncertainty. For a wide range of these problems the goal is to either quantify uncertainty and compute risk for critical events, optimize parameters or control strategies, and/or making decisions. Bayesian active learning provides a flexible framework for performing these tasks. However, Bayesian calculations are often prohibitively expensive in terms of the required simulations or experiments, even in the active learning setting. In this talk we introduce a new class of acquisition functions that utilize a likelihood-weighted ratio that accounts for the importance of the output relative to the input. This ratio acts essentially as a probabilistic sampling weight and guides the sampling algorithm towards regions of the input space where the objective function assumes abnormal values, resulting in significant savings of computational or experimental resources needed for convergence. In addition, the computational complexity of the likelihood-weighted acquisition functions is comparable with that of traditional active-learning approaches, and they can be approximated in a way that makes the approach tractable in high dimensions. We first show that the adopted acquisition functions can be rigorously derived as the asymptotic limit of an optimal acquisition function that has a minimax form over a functional space, i.e. it is optimal but not practical for computations. Subsequently, we demonstrate their favorable properties compared to standard methods on benchmark functions commonly used in the optimization community. Next, we apply the developed active learning framework on a series of real-world problems related to ocean and mechanical engineering. In particular, we consider the problem of uncertainty quantification of a prototype system for ship motions due to random waves, optimization of an experimental turbulent jet for maximum mixing, path planning for exploration of anomalies in unknown environments, and optimal sensor selection for detection of spatio-temporal temperature variability. Finally, we discuss future directions and challenges related to Bayesian methods and their application to physical and engineering problems.

Biography

Dr. Sapsis is Associate Professor of Mechanical and Ocean Engineering at MIT. He received a diploma in Ocean Engineering from Technical University of Athens, Greece and a Ph.D. in Mechanical and Ocean Engineering from MIT. Before becoming a faculty at MIT he was appointed Research Scientist at the Courant Institute of Mathematical Sciences at New York University. He has also been a visiting faculty at ETH-Zurich. Prof. Sapsis work lies on the interface of nonlinear dynamical systems, probabilistic modeling and data-driven methods. A particular emphasis of his work is the formulation of mathematical methods for the prediction, statistical quantification and optimization of complex engineering and physical systems such as turbulent fluid flows, nonlinear waves in the ocean, and extreme ship motions. He has received numerous awards and recognitions including three Young Investigator Awards (Navy, Army and Air-Force research office), the Alfred P. Sloan Foundation Award, and more recently the Verisk AI Faculty Research Award, the MathWorks Faculty Research Innovation Fellowship, the ASME Lloyd Hamilton Donnell Award and the Bodossaki Award on Basic Sciences: Mathematics.