Sequential Decision-Making in the Era of Intelligence: Safety, Resilience, and Non-Stationarity

ABSTRACT

This thesis advances the theory and practice of sequential decision-making in the era of intelligence, where algorithms increasingly act as primary decision-makers across digital platforms, supply chains, healthcare systems, and revenue-generating operations. While data-driven methods have achieved remarkable predictive performance, real-world intelligent systems must also ensure safety, resilience, and adaptability under non-stationarity — dimensions often overlooked when optimizing solely for expected short-term outcomes. The thesis develops new theoretical frameworks and algorithms addressing these challenges across online learning, supply chain management, and revenue management.

First, we study online learning with safety guarantees by analyzing the stochastic multi-armed bandit problem beyond expected regret. We characterize optimal regret tail behavior under optimal expectation guarantees, establishing tight upper and lower bounds on the expectation–risk trade-off. This analysis provides theoretical insight into why AlphaGo’s Monte Carlo Tree Search exploration bonus yields robust and reliable performance in high-stakes settings.

Second, we develop a framework for supply chain resilience under uncertain disruptions when Time-To-Recover (TTR) is unknown. We introduce a sequential prioritization model and a Risk Exposure Index based on expected lost sales. We derive tight competitive ratio bounds quantifying the cost of TTR uncertainty and characterize Pareto-optimal trade-offs between lost sales and inventory recovery.

Third, we address structured non-stationarity in revenue management through new models of budget-constrained allocation, recovering rewards, and reusable resources with decay. We propose algorithms achieving near-optimal regret or competitive ratios, substantially improving performance under dynamic conditions.

Together, these contributions provide new theoretical insights and algorithmic tools for designing intelligent decision systems that remain efficient, reliable, and robust under uncertainty, risk, and change.

COMMITTEE

David Simchi-Levi (advisor), John Tsitsiklis, Zeyu Zheng

EVENT INFORMATION

Hybrid event. To attend virtually, please contact the IDSS Academic Office (idss_academic_office@mit.edu) for connection information.