STA 414/2104 Winter 2023:

Statistical Methods for Machine Learning II

This course introduces probabilistic learning tools such as exponential families, directed graphical models, Markov random fields, exact inference techniques, message passing, sampling and mcmc, hidden Markov models, variational inference, EM algorithm, Bayesian regression, probabilistic PCA, Neural networks kernel methods, Gaussian processes, and variational autoencoders. It will also offer a broad view of model-building and optimization techniques that are based on probabilistic building blocks which will serve as a foundation for more advanced machine learning courses.

More details can be found in syllabus and piazza.



Prof Murat A. Erdogdu & Piotr Zwiernik
Office hours T 15:30 -17:30 (UY 9040)

Teaching Assistants:

Hossein Yousefi, Alireza Mousavi, Daniel Eftekhari, Madhu Gunasingam

Time & Location:

Section Room Lecture time Zoom link
STA 414 LEC0101 & STA 2104 LEC0101 MS 2172 M 14-17 link
STA 414 LEC5101 & STA 2104 LEC5101 MS 2172 T 18-21 link

Suggested Reading

No required textbooks. Suggested reading will be posted after each lecture (See lectures below).

Lectures and timeline

Week Lectures Recordings Suggested reading Tutorials Timeline
1 Introduction
Probabilistic Models
rec w1 MLPP 1 & 2
PRML 2.4
tut w1 syllabus
2 Decision theory
Directed Graphical Models
rec w2 PRML 1.5
tut w2  
3 Markov Random Fields
Exact inference
rec w3 MLPP 19-19.5, 20.3
ITIL 21.1, 26
tut w3 hw1 out
4 Message passing
Monte Carlo Methods
rec w4-1
rec w4-2
rec w4-t
MLPP 20.2,22.2
tut w4
bonus MP worksheet - tree
bonus MP worksheet - cycle
hw1 due
5 Sampling I
Sampling II
rec w5 MLPP 17.2, 24.3
this paper
tut w5
hw2 out
6 Hidden Markov Models
Variational inference I
rec w6 MLPP 17.3
MLPP 21.1-3
colab hw2 due
7 Reading week
(no class/tutorial)
- - - -
8 Midterm exam   practice midterm
- midterm
9 Variational inference II
EM algorithm
rec w9 Blei’s notes
PML2 10.1-10.2
PML1 3.5.1, 8.7.2-8.7.3
tut w9  
10 Probabilistic PCA
Bayesian regression
rec w10 PRML 12.2
PRML 3.3
tut w10 hw3 out
11 Kernel methods
Gaussian processes
rec w11 PRML 6.1-3
PRML 6.4
GP tutorial
tut w11
12 Neural Networks rec w12 notes NN tutorial hw3 due
13 Diffusion models
Final exam review
rec w13 more detailed diffusion blog -  

Computing Resources

For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options: