This course introduces commonly used machine learning algorithms such as linear and logistic regression, random forests, decision trees, neural networks, support vector machines, boosting etc. It will also offer a broad view of model-building and optimization techniques that are based on probabilistic building blocks which will serve as a foundation for more advanced machine learning courses.
The first half of the course focuses on supervised learning. We begin with nearest neighbours, decision trees, and ensembles. Then we introduce parametric models, including linear regression, logistic and softmax regression, and neural networks. We then move on to unsupervised learning, focusing in particular on probabilistic models, but also principal components analysis and K-means. More details can be found in syllabus and piazza.
Prof | Murat A. Erdogdu | Richard Zemel |
---|---|---|
csc311prof@cs.toronto.edu | csc311prof@cs.toronto.edu | |
Office hours | F 11-13 at Pratt 286B | W 13-14 Pratt 290G |
Cem Anil, Tianshi Cao, Harris Chan, Daniel Flam-Shepherd, Rasa Hosseinzadeh, Taylor Killian, Shun Liao, David Madras, Ehsan Mehralian, Bret Nestor, Nathan Ng, Anastasiia Razdaibiedina, Audrina Zhou
Section | Room | Lecture time | Tutorial time |
---|---|---|---|
L0101 | RW 117 | M 11-13 | M 15-16 |
L0201 | BA 1170 | W 11-13 | W 15-16 |
L0301 | MB 128 | Th 16-18 | Th 18-19 |
No required textbooks. Suggested reading will be posted after each lecture (See lectures below).
Week | Topics | Lectures | Tutorial materials | Suggested reading |
---|---|---|---|---|
1 | Introduction to ML & Nearest Neighbours | slides | Probability review slides & preliminaries | ESL 1. ESL 2.(1-3), 2.5 |
2 | Decision Trees | slides | Linear Alg review slides & knn demo | ESL 9.2 |
3 | Bias-Variance Decomposition Ensembles: Bagging Linear regression |
slides | Optimization and CV slides & notebooks | PRML 3.2 ESL 2.9, 8.7 ESL 3.2 |
4 | Linear Methods for Classification Optimization |
slides | Bias-Variance Decomposition notes & notebook |
PRML 4.3 ESL 4.(1,2,4) |
5 | Multiclass Classification Neural Networks |
slides | More on back propagation slides |
notes |
6 | Support Vector Machines Ensembles: Boosting |
slides | Linear Algebra & Midterm review slides | ESL 10.(1-5) |
7 | Probabilistic models | slides | No tutorial on Oct 17 & 23 | ESL 2.6.3, 6.6.3, 4.3.0 |
8 | Principal Component Analysis | slides | More on PCA slides & code & data | PRML 12.1 |
9 | k-Means & EM Algorithm | slides | More on k-Means & EM Algorithm slides & EM demo on Kaggle | PRML 9.1-2 |
10 | Matrix Factorizations & Recommender Systems | slides | More on Recommender Systems slides & Team Dinosaur Planet | ESL 14.5.1 |
11 | Reinforcement Learning | slides | Policy gradient slides & RL demo | RL 3, 4.1, 4.4, 6.1-6.5 |
12 | Algorithmic Fairness Final Exam Review |
slides | Final exam sample Q&A slides | Zemel et. al. Hardt et. al. |
Homework # | Out | Due | Materials | TA Office Hours |
---|---|---|---|---|
Homework 1 - V1 | Sep 18, 23:59 | Oct 5, 12:00 | questions data |
Sep 24 1-2pm & Sep 26 2-3pm & Sep 27 1-2pm @ BA3201 |
Homework 2 - V2 | Oct 3, 23:59 | Oct 17, 23:59 | questions q2_materials q3_materials |
Oct 9 2-3pm & Oct 11 2-3pm & Oct 16 2-3pm @ BA3289 |
Homework 3 - V4 | Oct 30, 2:59 | Nov 15, 23:59 | questions q1-2_materials q3_materials |
Nov 5 11am-12pm & Nov 7 1pm-2pm & Nov 11 4pm-5pm @ BA 3201 |
Homework 4 - V5 | Nov 20, 20:30 | Dec 5, 23:59 | questions q1-2_materials |
Nov 29th 10-11am & Nov 29th 3-4pm & Dec 2 12am-1pm @ BA3201 |
For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options:
The easiest option is probably to install everything yourself on your own machine.
If you don’t already have python, install it. We recommend using Anaconda. You can also install python directly if you know how.
conda create --name csc411
source activate csc411
pip install scipy numpy autograd matplotlib jupyter sklearn
All the required packages are already installed on the Teaching Labs machines.