Abstract: This talk reviews a theory of the functions learned by ReLU neural networks. At its core is the observation that deep ReLU networks can be characterized as solutions to data-fitting problems in certain infinite dimensional function spaces. The solutions are compositions of functions from Banach spaces of second-order bounded variation, defined in the Radon…
Abstract: One of the central goals in statistical learning theory is to understand which hypothesis classes can be learned from data. A landmark result in this direction is the Fundamental Theorem of PAC Learning, which characterizes the learnability of binary classification problems via the VC dimension, which was introduced by Vapnik and Chervonenkis in the…
Abstract: This talk traces the intellectual history of automated decision-making to its origins in the post-World War II development of computers. Mathematicians of the 1940s set out to design machines that could act as ideal rational agents in the face of uncertainty. In this pursuit, a cluster of foundational mathematical technologies—including linear programming, game theory,…
Abstract: Transformers have demonstrated remarkable chain-of-thought reasoning capabilities, yet, the underlying mechanisms by which they acquire and extrapolate these capabilities remain limited. This talk presents a theoretical analysis of transformers trained via gradient descent for symbolic reasoning and state tracking tasks with increasing problem complexity. Our analysis reveals the coordination of multi-head attention to solve…
Abstract: This talk reviews a theory of the functions learned by ReLU neural networks. At its core is the observation that deep ReLU networks can be characterized as solutions to data-fitting problems in certain infinite dimensional function spaces. The solutions are compositions of functions from Banach spaces of second-order bounded variation, defined in the Radon…
Abstract: One of the central goals in statistical learning theory is to understand which hypothesis classes can be learned from data. A landmark result in this direction is the Fundamental Theorem of PAC Learning, which characterizes the learnability of binary classification problems via the VC dimension, which was introduced by Vapnik and Chervonenkis in the…
Abstract: This talk traces the intellectual history of automated decision-making to its origins in the post-World War II development of computers. Mathematicians of the 1940s set out to design machines that could act as ideal rational agents in the face of uncertainty. In this pursuit, a cluster of foundational mathematical technologies—including linear programming, game theory,…
Abstract: Transformers have demonstrated remarkable chain-of-thought reasoning capabilities, yet, the underlying mechanisms by which they acquire and extrapolate these capabilities remain limited. This talk presents a theoretical analysis of transformers trained via gradient descent for symbolic reasoning and state tracking tasks with increasing problem complexity. Our analysis reveals the coordination of multi-head attention to solve…
