This course covers several topics in classical machine learning theory.
Topics may include: Asymptotic statistics, Uniform Convergence, Generalization, Complexity measures, Kernel Methods, Online Learning, Sampling. More details can be found in syllabus. Please also sign up for Piazza.
This class requires a good informal knowledge of probability theory, linear algebra, real analysis (at least Masters level). Homework 0 is a good way to check your background.
No required textbooks. Suggested reading will be posted after each lecture (See lectures below).
|Week||Day||Topics and Lecture notes||Lectures||Recordings||Timeline|
|1||1/14||Introduction & Warm-up: Gaussian Mean Estimation||notes 1||lecture 1||syllabus|
|2||1/21||Exponential Families and Information Inequality|
|3||1/28||Asymptotic statistics||hw1 out|
|4||2/04||Uniform convergence & Generalization|
|5||2/11||Covering with epsilon-nets||hw1 due & hw2 out|
|6||2/18||Rademacher complexity I|
|7||2/25||Rademacher complexity II||hw2 due|
|8||3/04||Combinatorial Measures of Complexity||hw3 out|
|9||3/11||Chaining and Dudley’s theorem||project proposal due|
|10||3/18||Algorithmic Stability||hw 3 due|
|12||4/01||Kernel Methods I|
|13||4/08||Kernel Methods II||Final reports due|
|Homework #||Out||Due||TA Office Hours|
|Homework 0 - V0||1/11||-||-|
|Homework 1 - V0||1/28||2/11||-|
|Homework 2 - V0||2/11||2/25||-|
|Homework 3 - V0||3/01||3/18||-|
Latex template can be found here.
Your project goal is to read and write a comprehensive review of a theoretical machine learning paper, and understand the main building blocks.
Project Inspiration: You can go through recent papers on COLT, NeurIPS, ICML, ICLR, JMLR to get project ideas and pick a paper to review.
List of suggested papers will be posted here.
Latex template for reports can be found here.
pip install scipy numpy matplotlib jupyter sklearn