CSC 2532 Winter 2022: Statistical Learning Theory

This course covers several topics in classical machine learning theory.

Topics may include: Asymptotic statistics, Uniform Convergence, Generalization, Complexity measures, Kernel Methods, Online Learning, Sampling. More details can be found in syllabus. Please also sign up for Piazza.

This class requires a good informal knowledge of probability theory, linear algebra, real analysis (at least Masters level). Homework 0 is a good way to check your background.


Announcements:


Instructors: Murat A. Erdogdu

Teaching Assistants: Mert Vural, Matthew Zhang


Time & Location:

Section Room Lecture time
L0101 online F 16-18

Suggested Reading

No required textbooks. Suggested reading will be posted after each lecture (See lectures below).


Lectures and timeline

Week Day Topics and Lecture notes Lectures Recordings Timeline
1 1/14 Introduction & Warm-up: Gaussian Mean Estimation notes 1 lecture 1 syllabus
2 1/21 Exponential Families and Information Inequality      
3 1/28 Asymptotic statistics     hw1 out
4 2/04 Uniform convergence & Generalization      
5 2/11 Covering with epsilon-nets     hw1 due & hw2 out
6 2/18 Rademacher complexity I      
7 2/25 Rademacher complexity II     hw2 due
8 3/04 Combinatorial Measures of Complexity     hw3 out
9 3/11 Chaining and Dudley’s theorem     project proposal due
10 3/18 Algorithmic Stability     hw 3 due
11 3/25 Midterm      
12 4/01 Kernel Methods I      
13 4/08 Kernel Methods II     Final reports due

Homeworks

Homework # Out Due TA Office Hours
Homework 0 - V0 1/11 - -
Homework 1 - V0 1/28 2/11 -
Homework 2 - V0 2/11 2/25 -
Homework 3 - V0 3/01 3/18 -

Latex template can be found here.


Project

Your project goal is to read and write a comprehensive review of a theoretical machine learning paper, and understand the main building blocks.

Project Inspiration: You can go through recent papers on COLT, NeurIPS, ICML, ICLR, JMLR to get project ideas and pick a paper to review.

List of suggested papers will be posted here.

Latex template for reports can be found here.

Computing Resources

For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn.