CS 229 Machine Learning Course Materials

Handouts and Problem Sets

• Handout #1: Course Information (HTML)
• Handout #2: Course Schedule (HTML)

Lecture Notes

• Lecture notes 1 (ps) (pdf) Supervised Learning, Discriminative Algorithms

• Lecture notes 2 (ps) (pdf) Generative Algorithms

• Lecture notes 3 (ps) (pdf) Support Vector Machines

• Lecture notes 4 (ps) (pdf) Learning Theory

• Lecture notes 5 (ps) (pdf) Regularization and Model Selection

• Lecture notes 6 (ps) (pdf) Online Learning and the Perceptron Algorithm. (optional reading)

• Lecture notes 7a (ps) (pdf) Unsupervised Learning, k-means clustering.

• Lecture notes 7b (ps) (pdf) Mixture of Gaussians

• Lecture notes 8 (ps) (pdf) The EM Algorithm

• Lecture notes 9 (ps) (pdf) Factor Analysis

• Lecture notes 10 (ps) (pdf) Principal Components Analysis

• Lecture notes 11 (ps) (pdf) Independent Components Analysis

• Lecture notes 12 (ps) (pdf) Reinforcement Learning and Control

Supplemental Notes

• Supplemental notes 1 (pdf) Binary classification with +/-1 labels.

• Supplemental notes 2 (pdf) Boosting algorithms and weak learning.

  • Matlab code to generate plots (.m) Functional after implementing stump_booster.m in PS2.

• Supplemental notes 3 (pdf) The representer theorem.

• Supplemental notes 4 (pdf) Hoeffding’s inequality.

Section Notes

• Section notes 1 (pdf) Linear Algebra Review and Reference

• Section notes 2 (pdf) Probability Theory Review

• Section notes 3 (pdf) Files for the Matlab tutorial: sigmoid.m, logistic_grad_ascent.m, matlab_session.m

• Section notes 4 (ps) (pdf) Convex Optimization Overview, Part I

• Section notes 5 (ps) (pdf) Convex Optimization Overview, Part II

• Section notes 6 (ps) (pdf) Hidden Markov Models

• Section notes 7 (pdf) The Multivariate Gaussian Distribution

• Section notes 8 (pdf) More on Gaussian Distribution

• Section notes 9 (pdf) Gaussian Processes

Other resources

Advice on applying machine learning: Slides from Andrew’s lecture on getting machine learning algorithms to work in practice can be found here.

Previous projects: A list of last year’s final projects can be found here.

Matlab resources:
Here are a couple of Matlab tutorials that you might find helpful: http://www.math.ucsd.edu/~bdriver/21d-s99/matlab-primer.html and http://www.math.mtu.edu/~msgocken/intro/node1.html. For emacs users only: If you plan to run Matlab in emacs, here are matlab.el, and a helpful .emac’s file.

Octave resources:
For a free alternative to Matlab, check out GNU Octave. The official documentation is available here. Some useful tutorials on Octave include http://en.wikibooks.org/wiki/Octave_Programming_Tutorial and http://www-mdp.eng.cam.ac.uk/web/CD/engapps/octave/octavetut.pdf

Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NIPS (all old NIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.

Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don’t already have one.