Abstract: Modern large language models have billions to trillions of parameters, creating enormous computational and memory costs. Quantization, i.e. reducing their numerical precision, is the leading practical mitigation strategy. But how far can we push it, and what do we lose? This talk addresses different sides of this question. First, for post-training quantization, we characterize the accuracy–compression frontier focusing on large-scale evaluations and new formats. Second, for quantization-aware training, we show that convergence behavior is predicted by representation scaling laws,…

TBD

TBD

TBD

TBD

Abstract: The emergence of large language models has prompted a surge of interest into theoretical models that might give us insight into both their successes and their shortcomings. We'll give an overview of recent work in this direction, focusing on a surprising line of positive results that shows it is possible to give guarantees for language-generation algorithms even in the absence of any probabilistic assumptions, in a framework known as "language generation in the limit". These results suggest interesting notions…