STA 414/2104 Winter 2023:

Statistical Methods for Machine Learning II

This course introduces probabilistic learning tools such as exponential families, directed graphical models, Markov random fields, exact inference techniques, message passing, sampling and mcmc, hidden Markov models, variational inference, EM algorithm, Bayesian regression, probabilistic PCA, Neural networks kernel methods, Gaussian processes, and variational autoencoders. It will also offer a broad view of model-building and optimization techniques that are based on probabilistic building blocks which will serve as a foundation for more advanced machine learning courses.

More details can be found in syllabus and piazza.

Announcements:

Final exam is in-person on 24 Apr 9am-12pm at BN 322. Practice final exam and further info is here.
Hw 3 is out, and due on 03/27 23:59. TA office hours on 03/22 & 03/23, 1-2pm at UY9040.
Midterm logistics can be found here. Hw1 solutions are posted on quercus.
OHs will be online during the reading week. Zoom link.
A representative practice midterm is released with solutions.
Hw 2 deadline extended to Feb 22, 23:59.
Hw 2 is out, and due on 02/19 23:59. TA office hours 2/14 & 2/15 14-15 online link.
Hw 1 is out, and due on 02/05 23:59. TA office hours are on 1/30 & 2/01, 12:30-13:30 at Hydro 9013.
Instructor OHs are online this week (01/17).
Lectures begin on Jan 9!

Instructors:

Prof	Murat A. Erdogdu & Piotr Zwiernik
Email	sta414-2104prof@cs.toronto.edu
Office hours	T 15:30 -17:30 (UY 9040)

Teaching Assistants:

Hossein Yousefi, Alireza Mousavi, Daniel Eftekhari, Madhu Gunasingam

Email: sta414-2104ta@cs.toronto.edu

Time & Location:

Section	Room	Lecture time	Zoom link
STA 414 LEC0101 & STA 2104 LEC0101	MS 2172	M 14-17	link
STA 414 LEC5101 & STA 2104 LEC5101	MS 2172	T 18-21	link

Lectures and timeline

Week	Lectures	Recordings	Suggested reading	Tutorials	Timeline
1	Introduction Probabilistic Models	rec w1	MLPP 1 & 2 PRML 2.4	tut w1	syllabus
2	Decision theory Directed Graphical Models	rec w2	PRML 1.5 MLPP 10	tut w2
3	Markov Random Fields Exact inference	rec w3	MLPP 19-19.5, 20.3 ITIL 21.1, 26	tut w3	hw1 out
4	Message passing Monte Carlo Methods	rec w4-1 rec w4-2 rec w4-t	MLPP 20.2,22.2 ITIL 29	tut w4 bonus MP worksheet - tree bonus MP worksheet - cycle	hw1 due
5	Sampling I Sampling II	rec w5	MLPP 17.2, 24.3 this paper	tut w5 j-notebook	hw2 out
6	Hidden Markov Models Variational inference I	rec w6	MLPP 17.3 MLPP 21.1-3	colab	hw2 due
7	Reading week (no class/tutorial)	-	-	-	-
8	Midterm exam		practice midterm solutions	-	midterm
9	Variational inference II EM algorithm	rec w9	Blei’s notes PML2 10.1-10.2 PML1 3.5.1, 8.7.2-8.7.3	tut w9
10	Probabilistic PCA Bayesian regression	rec w10	PRML 12.2 PRML 3.3	tut w10	hw3 out
11	Kernel methods Gaussian processes	rec w11	PRML 6.1-3 PRML 6.4	GP tutorial tut w11
12	Neural Networks	rec w12	notes	NN tutorial	hw3 due
13	Diffusion models Final exam review	rec w13	more detailed diffusion blog	-

Computing Resources

For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options:

The easiest option is run everything on colab.
Alternatively, you can install everything yourself on your own machine.
- If you don’t already have python, install using Anaconda.
- Use pip to install the required packages pip install scipy numpy autograd matplotlib jupyter sklearn
For those unfamiliar with Numpy, there are many good resources, e.g. Numpy tutorial and Numpy Quickstart.