This course covers several topics in classical machine learning theory.

Topics may include: Asymptotic statistics, Uniform Convergence, Generalization, Complexity measures, Kernel Methods, Feature Learning, Neural Networks. More details can be found in syllabus. Please also sign up for Piazza.

This class requires a good informal knowledge of probability theory, linear algebra, real analysis (at least Masters level). Homework 0 is a good way to check your background.

- HW2 is out and due on 2/19 23:59, to be submitted through crowdmark. TA OHs on 02/16 1pm @ Pratt 278.
- Lectures will be hybrid in room SU 255 and streamed over zoom.
- HW1 is out and due on 2/04 23:59, to be submitted through crowdmark. TA OHs on 02/02 1pm @ Pratt 278.

- Email: csc2532prof@cs.toronto.edu
- Office hours: W 11:00-12:00 at Pratt 286b

- Email: csc2532tas@cs.toronto.edu

Section | Room | Lecture time | Zoom |
---|---|---|---|

L0101 | SU 255 | Th 16-18 | link |

Week | Topics and Lecture notes | Board | Old Recordings | Timeline |
---|---|---|---|---|

1 | Uniform convergence & Generalization | notes 1 | lecture 1 | syllabus |

2 | Covering with epsilon-nets | notes 2 | lecture 2 | |

3 | Symmetrization | notes 3 | lecture 3 | hw1 out |

4 | Rademacher complexity | notes 4 | lecture 4 | |

5 | Combinatorial Measures of Complexity | notes 5 | lecture 5 | hw1 due & hw2 out |

6 | Chaining | notes 6 | lecture 6 | |

7 | Reading week | hw2 due | ||

8 | Kernel Methods I | notes 7 | lecture 7 | project proposal due |

9 | Kernel Methods II | notes 8 | lecture 8 | |

10 | Midterm | |||

11 | Double-descent Risk Curves | hw 3 out | ||

12 | Neural Networks: Linearization | |||

13 | Neural Networks: Feature Learning | hw 3 due |

Homework # | Out | Due | TA Office Hours |
---|---|---|---|

Homework 0 - V0 | 1/11 | - | - |

Homework 1 - V0 | 1/22 | 2/04 23:59 | 02/02 1pm @ Pratt 278 |

Homework 2 - V0 | 2/06 | 2/19 23:59 | 02/16 1pm @ Pratt 278 |

Latex template can be found here.

Final project should give you experience on carrying out theoretical research.

More details TBA.

For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options:

- The easiest option is run everything on colab.
- Alternatively, you can install everything yourself on your own machine.
- If you don’t already have python, install using Anaconda.
- Use pip to install the required packages
`pip install scipy numpy autograd matplotlib jupyter sklearn`

- For those unfamiliar with Numpy, there are many good resources, e.g. Numpy tutorial and Numpy Quickstart.