MACHINE LEARNING January 20, 2009

COMS4771-001 COURSE INFO

Day & Time and Location	T/Th 2:40pm-3:55pm at 833 Mudd
Instructor	Professor Tony Jebara, jebara(at)cs(dot)columbia(dot)edu
Office Hours	CEPSR 605, T/Th 4:00pm-4:45pm
TAs	Thierry Bertin-Mahieux tb2332(at)columbia(dot)edu Haoyun Feng hf2172(at)columbia(dot)edu Olivier Cossairt, ollie(at)cs(dot)columbia(dot)edu
Bulletin Board	Available via courseworks.columbia.edu and is the best method of contact between students and Professor/TAs for general questions that would be relevant for the whole class (i.e. clarifications on lectures, questions about homework, etc.).

Prerequisites: Knowledge of linear algebra and introductory probability or statistics.

Description: This course introduces topics in machine learning for both generative
and discriminative estimation. Material will include least squares methods, Gaussian
distributions, linear classification, linear regression, maximum likelihood, exponential
family distributions, Bayesian networks, Bayesian inference, mixture models, the EM
algorithm, graphical models, hidden Markov models, support vector machines, and
kernel methods. Students are expected to implement several algorithms in Matlab
and have some background in linear algebra and statistics.

Required Texts:

Michael I. Jordan and Christopher M. Bishop, Introduction to Graphical Models.

Still unpublished. Available online (password-protected) on class home page.

Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer.

2006 First Edition is preferred. ISBN: 0387310738. 2006.

Optional Texts: Available at library (additional handouts will also be given).

Tony Jebara, Machine Learning: Discriminative and Generative, Kluwer, 2004

ISBN: 1-4020-7647-9. Boston, MA, 2004.

R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification, John Wiley & Sons, 2001.

Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical

Learning. Springer Series in Statistics, Springer-Verlag New York USA. 2001.

Tom M. Mitchell, Machine Learning, McGraw-Hill Series in Computer Science, 1997.

Graded Work: Grades will be based on 5 homeworks (about 50%), the midterm (about

20%) and the final exam. (about 30%). Any material covered in assigned book

readings, handouts, homework, lectures or discussion sections may appear in exam

The midterm is the last class before Spring Break.

If you miss the midterm and don't have an official reason, you will get 0 on it.

If you have an official reason, your midterm grade will be based on the final exam.

Tentative Schedule:

Date	Topic
January 20	Lecture 01: Introduction
January 22	Lecture 02: Least Squares
January 27	Lecture 03: Linear Classification and Regression
January 29	Lecture 04: Neural Networks and BackProp
February 3	Lecture 05: Neural Networks and BackProp
February 5	Lecture 06: Support Vector Machines
February 10	Lecture 07: Support Vector Machines
February 12	Lecture 08: Kernels
February 17	Lecture 09: Probability Models
February 19	Lecture 10: Probability Models
February 24	Lecture 11: Probability Models, Bernoulli
February 26	Lecture 12: Naive Bayes, Multinomials, Text
March 3	Lecture 13: Graphical Models Preview
March 5	Lecture 14: Gaussians, Estimation, Sampling
March 10	Lecture 15: Gaussian Classifiers, Regressors, PCA
March 12	MIDTERM
March 17	SPRING BREAK
March 19	SPRING BREAK
March 24	Lecture 16: Bayesian Inference
March 26	Lecture 17: The Exponential Family
March 31	Lecture 18: Mixtures, K-Means and Clustering
April 2	Lecture 19: Expectation Maximization
April 7	Lecture 20: EM and Extensions
April 9	Lecture 21: Graphical Models
April 14	Lecture 22: Graphical Models
April 16	Lecture 23: Support Vector Machines
April 21	Lecture 24: Junction Tree Algorithm
April 23	Lecture 25: Junction Tree Algorithm
April 28	Lecture 26: Hidden Markov Models
April 30	Lecture 27: HMMs and Structure Learning

Class Attendance: You are responsible for all material presented in the class

lectures, recitations, and so forth. Some material will diverge from the textbooks

so regular attendance is important.

Late Policy: If you hand in late work without approval of the instructor or TAs,

you will receive zero credit. Homework is due at the beginning of class on the

due date.

Cooperation on Homework: Collaboration on solutions, sharing or copying of

solutions is not allowed. Of course, no cooperation is allowed during exams.

This policy will be strictly enforced.

Web Page: The class URL is: http://www.cs.columbia.edu/~jebara/4771 and

will contain copies of handouts, homework assignments, solutions and other

information.

Computer Accounts: You will need an ACIS computer account for email, use

of Matlab (unless you have a windows version) and so forth.