Abstract: Deep learning algorithms operate in regimes that defy classical learning theory. Neural networks architectures often contain more parameters than training samples. Despite their huge complexity, the generalization error achieved on real data is small. In this talk, we aim to study generalization properties of algorithms in high dimensions. Interestingly, we show that algorithms in high dimension require a small bias for good generalization. We show that this is indeed the case for deep neural networks in the overparameterized regime. In addition, we provide lower bounds on the generalization error in various settings for any algorithm. We calculate such bounds using random matrix theory (RMT). We will review the connection between deep neural networks and RMT and existing results. These bounds are particularly useful when the analytic evaluation of standard performance bounds is not possible due to the complexity and nonlinearity of the model. The bounds can serve as a benchmark for testing performance and optimizing the design of actual learning algorithms.
(Joint work with Prof. Ofer Zeitouni)

This is the third lecture in the series. Details can be found here: Algebra and Geometry lecture series (yale.edu)

Abstract:

We study the moduli space of unit-volume Einstein 4-manifolds near its finite-distance boundary, that is, the noncollapsed singularity formation. We prove that any smooth Einstein 4-manifold close to a singular one in a mere Gromov-Hausdorff (GH) sense is the result of a gluing-perturbation procedure that we develop and which handles the presence of multiple trees of singularities at arbitrary scales. This sheds some light on the structure of the moduli space and lets us show that spherical and hyperbolic orbifolds which are Einstein in a synthetic sense cannot be GH-approximated by smooth Einstein metrics.