CSC 311 Fall 2019: Introduction to Machine Learning
This course introduces commonly used machine learning algorithms such as linear and logistic regression, random forests, decision trees, neural networks, support vector machines, boosting etc. It will also offer a broad view of model-building and optimization techniques that are based on probabilistic building blocks which will serve as a foundation for more advanced machine learning courses.
The first half of the course focuses on supervised learning. We begin with nearest neighbours, decision trees, and ensembles. Then we introduce parametric models, including linear regression, logistic and softmax regression, and neural networks. We then move on to unsupervised learning, focusing in particular on probabilistic models, but also principal components analysis and K-means. More details can be found in syllabus and piazza.
Announcements:
- Final exam in on Fri, Dec 13, 2019 9-12, at EX 200
- Final exam covers everything in lectures 1 through 11. It doesn’t cover materials only covered in tutorials.
- Practice final exam and its solutions are posted.
- No office hours during the final exam week since all instructors are at NeurIPS.
Instructors:
| Prof | Murat A. Erdogdu | Richard Zemel |
|---|---|---|
| csc311prof@cs.toronto.edu | csc311prof@cs.toronto.edu | |
| Office hours | F 11-13 at Pratt 286B | W 13-14 Pratt 290G |
Teaching Assistants:
Cem Anil, Tianshi Cao, Harris Chan, Daniel Flam-Shepherd, Rasa Hosseinzadeh, Taylor Killian, Shun Liao, David Madras, Ehsan Mehralian, Bret Nestor, Nathan Ng, Anastasiia Razdaibiedina, Audrina Zhou
- Email: csc311ta@cs.toronto.edu
Time & Location:
| Section | Room | Lecture time | Tutorial time |
|---|---|---|---|
| L0101 | RW 117 | M 11-13 | M 15-16 |
| L0201 | BA 1170 | W 11-13 | W 15-16 |
| L0301 | MB 128 | Th 16-18 | Th 18-19 |
Suggested Reading
No required textbooks. Suggested reading will be posted after each lecture (See lectures below).
- (PRML) Christopher M. Bishop (2006) Pattern Recognition and Machine Learning
- (DL) Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016), Deep Learning
- (MLPP) Kevin P. Murphy (2013), Machine Learning: A Probabilistic Perspective
- (ESL) Trevor Hastie, Robert Tibshirani, Jerome Friedman (2009) The Elements of Statistical Learning
- (ISL) Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani (2017) Introduction to Statistical Learning
- (ITIL) David MacKay (2003) Information Theory, Inference, and Learning Algorithms
- (RL) Richard S. Sutton, Andrew G. Barto (2018) Reinforcement Learning: An Introduction
Lectures
| Week | Topics | Lectures | Tutorial materials | Suggested reading |
|---|---|---|---|---|
| 1 | Introduction to ML & Nearest Neighbours | slides | Probability review slides & preliminaries | ESL 1. ESL 2.(1-3), 2.5 |
| 2 | Decision Trees | slides | Linear Alg review slides & knn demo | ESL 9.2 |
| 3 | Bias-Variance Decomposition Ensembles: Bagging Linear regression |
slides | Optimization and CV slides & notebooks | PRML 3.2 ESL 2.9, 8.7 ESL 3.2 |
| 4 | Linear Methods for Classification Optimization |
slides | Bias-Variance Decomposition notes & notebook |
PRML 4.3 ESL 4.(1,2,4) |
| 5 | Multiclass Classification Neural Networks |
slides | More on back propagation slides |
notes |
| 6 | Support Vector Machines Ensembles: Boosting |
slides | Linear Algebra & Midterm review slides | ESL 10.(1-5) |
| 7 | Probabilistic models | slides | No tutorial on Oct 17 & 23 | ESL 2.6.3, 6.6.3, 4.3.0 |
| 8 | Principal Component Analysis | slides | More on PCA slides & code & data | PRML 12.1 |
| 9 | k-Means & EM Algorithm | slides | More on k-Means & EM Algorithm slides & EM demo on Kaggle | PRML 9.1-2 |
| 10 | Matrix Factorizations & Recommender Systems | slides | More on Recommender Systems slides & Team Dinosaur Planet | ESL 14.5.1 |
| 11 | Reinforcement Learning | slides | Policy gradient slides & RL demo | RL 3, 4.1, 4.4, 6.1-6.5 |
| 12 | Algorithmic Fairness Final Exam Review |
slides | Final exam sample Q&A slides | Zemel et. al. Hardt et. al. |
Homeworks
| Homework # | Out | Due | Materials | TA Office Hours |
|---|---|---|---|---|
| Homework 1 - V1 | Sep 18, 23:59 | Oct 5, 12:00 | questions data |
Sep 24 1-2pm & Sep 26 2-3pm & Sep 27 1-2pm @ BA3201 |
| Homework 2 - V2 | Oct 3, 23:59 | Oct 17, 23:59 | questions q2_materials q3_materials |
Oct 9 2-3pm & Oct 11 2-3pm & Oct 16 2-3pm @ BA3289 |
| Homework 3 - V4 | Oct 30, 2:59 | Nov 15, 23:59 | questions q1-2_materials q3_materials |
Nov 5 11am-12pm & Nov 7 1pm-2pm & Nov 11 4pm-5pm @ BA 3201 |
| Homework 4 - V5 | Nov 20, 20:30 | Dec 5, 23:59 | questions q1-2_materials |
Nov 29th 10-11am & Nov 29th 3-4pm & Dec 2 12am-1pm @ BA3201 |
Computing Resources
For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. You have two options:
-
The easiest option is probably to install everything yourself on your own machine.
-
If you don’t already have python, install it. We recommend using Anaconda. You can also install python directly if you know how.
- Optionally, create a virtual environment for this class and step into it. If you have a conda distribution run the following commands:
conda create --name csc411 source activate csc411 - Use pip to install the required packages
pip install scipy numpy autograd matplotlib jupyter sklearn
-
-
All the required packages are already installed on the Teaching Labs machines.