SCS Seminar Talk: Zhao Song

TITLE: The Theory of Provable and Efficient Deep Learning 

ABSTRACT:

Deep learning has fundamentally revolutionized our society, including recommendation systems that suggest movies and items to purchase, virtual assistants, and  future autonomous vehicles. From a theoretical point of view, there are two main questions in deep learning: (1) understanding of why deep learning works, (2) how to speed the running time of deep learning algorithms. In this talk, I will provide progress for these two questions. First, I will show that stochastic gradient descent is able to memorize all the input data points under some mild assumptions. Second, I will present a new optimization algorithm for fast deep learning. The key observation is that stochastic gradient descent is an iterative algorithm, and gradients change slowly in each iteration. This observation allows us to update gradients in a lazy fashion, significantly improving per-iteration running time. Finally, I will briefly describe how to apply similar ideas to speed up convex optimization and quantum machine learning.

BIO:

 Zhao Song is a postdoctoral researcher at Princeton University and Institute for Advanced Study. He obtained his Ph.D. from the Computer Science Department at University of Texas at Austin under the supervision of Professor Eric Price in 2019. His Ph.D. thesis won Bert Kay Dissertation Award (the best dissertation award) at the Computer Science Department in the University of Texas at Austin. His recent work, InstaHide, won second place for the Bell Lab Prize, which is awarded to the top-3 most innovative technology breakthroughs annually.

JOIN TALK HERE: https://bluejeans.com/946032411