CSC 412/2506 Winter 2024:

Probabilistic Machine Learning

This course introduces probabilistic learning tools such as exponential families, directed graphical models, Markov random fields, exact inference techniques, message passing, sampling and mcmc, hidden Markov models, variational inference, EM algorithm, Bayesian regression, probabilistic PCA, Neural networks kernel methods, Gaussian processes, variational autoencoders, and diffusion models. It will also offer a broad view of model-building and optimization techniques that are based on probabilistic building blocks which will serve as a foundation for more advanced machine learning courses.

More details can be found in syllabus and piazza.

Announcements:

Final exam is in-person on 18 Apr 7pm-10pm in EX 310 (A-DE) and EX 320 (DEN-Z). Practice final exam and further info is here.
No tutorials on 3/29 since the University is closed.
A3 deadline is extended to 3/31 23:59.
A3 is out, and due on 3/26 23:59 (start early). TA office hours are 3/21 10-11, 3/22 15-16 at Pratt 286.
You may submit A2 by Feb 22, without any penalty.
A representative practice midterm is released along with its solutions.
Midterm logistics can be found here. A1 solutions are posted on quercus.
A2 is out, and due on 02/18 23:59. TA office hours are 02/15 15-16, 02/16 17-18 at BA2270.
A1 is out, and due on 02/04 23:59. TA office hours are 01/30 12-13, 02/01 13-14 at BA2270.
Lectures begin on Jan 8!

Instructors:

Prof	Murat A. Erdogdu
Email	csc412prof@cs.toronto.edu
Office hours	W 15-17 @Pratt 286b

Teaching Assistants:

Vahid Balazadeh, Daniel Eftekhari, Alireza Keshavarzian, Alireza Mousavi (Head TA), Mert Vural, Haoping Xu, Matthew Zhang

Email: csc412ta@cs.toronto.edu

Time & Location:

Section	Lecture	Tutorial
CSC412 LEC0101-2001 & CSC2506 LEC0101	M 13-15 @ GB 119	F 13-14 @ GB 119
CSC412 LEC0201 & CSC2506 LEC0201	W 13-15 @ MP 137	F 14-15 @ ES B142

Lectures and timeline

Week	Lectures	Recordings	Suggested reading	Tutorials	Timeline
1	Introduction Probabilistic Models	lec01 tut01	MLPP 1 & 2 PRML 2.4	tut01	syllabus
2	Decision theory Directed Graphical Models	lec02 tut02	PRML 1.5 MLPP 10	tut02
3	Markov Random Fields Exact inference	lec03 tut03	MLPP 19-19.5, 20.3 ITIL 21.1, 26	tut03	A1 out
4	Message passing Monte Carlo Methods	lec04 tut04	MLPP 20.2,22.2 ITIL 29	tut04	A1 due
5	Sampling I Sampling II	lec05 tut05	MLPP 17.2, 24.3 this paper	colab demos	A2 out
6	Hidden Markov Models Variational inference I	lec06 tut06	MLPP 17.3 MLPP 21.1-3	colab	A2 due
7	Reading week (no class/tutorial)	-	-	-	-
8	Midterm exam				midterm
9	Variational inference II Variational Autoencoders	lec09 tut09	PRML 10.1-10.2 Blei’s notes	colab	A3 out
10	EM algorithm Probabilistic PCA	lec10 tut10	PRML 9 PRML 12.2	colab
11	Bayesian regression Kernel methods	lec11 tut11	PRML 6.1-3 PRML 3.3	colab	A3 due
12	Gaussian processes Diffusion models	lec12	PRML 6.4 CVPR tutorial this blog	GP tutorial
13	Diffusion models II Final exam review	lec13 tut13	Exam details

Assignments

Assignment #	Out	Due	TA Office Hours
Assignment 1	1/22	2/04	1/30 12-13, 2/01 13-14 at BA2270
Assignment 2	2/05	2/18	2/15 15-16, 2/16 17-18 at BA2270
Assignment 3	3/05	3/26	3/21 10-11, 3/22 15-16 at Pratt286

Computing Resources

For the assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options:

The easiest option is run everything on colab.
Alternatively, you can install everything yourself on your own machine.
- If you don’t already have python, install using Anaconda.
- Use pip to install the required packages pip install scipy numpy autograd matplotlib jupyter sklearn
For those unfamiliar with Numpy, there are many good resources, e.g. Numpy tutorial and Numpy Quickstart.