CSC 2532 Winter 2024: Statistical Learning Theory

This course covers several topics in classical machine learning theory.

Topics may include: Asymptotic statistics, Uniform Convergence, Generalization, Complexity measures, Kernel Methods, Feature Learning, Neural Networks. More details can be found in syllabus. Please also sign up for Piazza.

This class requires a good informal knowledge of probability theory, linear algebra, real analysis (at least Masters level). Homework 0 is a good way to check your background.


Announcements:


Instructors: Murat A. Erdogdu

Teaching Assistants: Mert Vural


Time & Location:

Section Room Lecture time Zoom
L0101 SU 255 Th 16-18 link

Lectures and timeline

Week Topics and Lecture notes Board Old Recordings Timeline
1 Uniform convergence & Generalization notes 1 lecture 1 syllabus
2 Covering with epsilon-nets notes 2 lecture 2  
3 Symmetrization notes 3 lecture 3 hw1 out
4 Rademacher complexity notes 4 lecture 4  
5 Combinatorial Measures of Complexity notes 5 lecture 5 hw1 due & hw2 out
6 Chaining notes 6 lecture 6  
7 Reading week     hw2 due
8 Kernel Methods I notes 7 lecture 7 project proposal due
9 Kernel Methods II notes 8 lecture 8  
10 Midterm      
11 Double-descent Risk Curves notes 9   hw 3 out
12 Neural Networks: Linearization notes 10    
13 Neural Networks: Feature Learning notes 11   hw 3 due

Homeworks

Homework # Out Due TA Office Hours
Homework 0 - V0 1/11 - -
Homework 1 - V0 1/22 2/04 23:59 02/02 1pm @ Pratt 278
Homework 2 - V0 2/06 2/19 23:59 02/16 1pm @ Pratt 278
Homework 3 - V0 3/20 4/05 23:59  

Latex template can be found here.


Project

Final project should give you experience on carrying out theoretical research.

Evaluation will be based on two reports:

You may work on the project alone or in a group of 2. Groups of 2 need to review 2 papers and the standards for a group project will be higher.

Project Inspiration: You can go through recent papers on COLT, NeurIPS, ICML, ICLR, JMLR to get project ideas and pick a paper to review.

Several research directions can be found here, but the list is by no means comprehensive.

Latex template for reports can be found here.

Computing Resources

For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options: