CSCI-GA 3033 - Honors Machine Learning - Fall 2026
Logistics
- Instructor: Gautam Kamath
- Course Communication: TBD.
- Lecture: Tuesdays, 4:55 PM - 6:55 PM.
- Location: Silv 401.
- Midterm Exam: October 20.
- Final Exam: TBD.
- Office Hours: TBD.
- Teaching Assistants: TBD.
- Syllabus: TBD.
Course Description
An advanced introduction to machine learning. This course is intended for PhD students who wish to conduct technical research in machine learning. It will consequently have higher expectations in terms of preparation, and aim to leave you with a deeper understanding of the material. It is intended to serve as a foundation for research in machine learning as well as a prerequisite for more advanced machine learning courses.
Topics will include learning fundamentals (loss functions, empirical risk minimization, overfitting, regularization, maximum likelihood estimation, and maximum a posteriori estimation); optimization fundamentals (gradient descent and stochastic gradient descent); classical supervised learning methods (linear regression, k-nearest neighbors, logistic regression, support vector machines, kernels, decision trees, boosting, and bagging); unsupervised learning methods (principal component analysis, k-means clustering, and Gaussian mixture models); and neural networks (multilayer perceptrons, deep networks, convolutional neural networks, and generative adversarial networks).
This is the first time this course is being offered, and thus details are subject to change.
Prerequisites
Undergraduate courses in linear algebra and strong programming skills for implementation of algorithms studied in class. Strong knowledge of probability.
Recommended: Knowledge of vector calculus, elementary statistics, and MATH-UA 120: Discrete Mathematics.
This course versus CSCI-GA.2565
This course and CSCI-GA.2565 are offered at the exact same time, and are both graduate-level introductions to machine learning. It is natural to wonder what the difference is, and who should take which course. You should take this course if you are an ML PhD student and/or intend to conduct research in machine learning and you wish to have a deeper understanding of the foundational topics.
Lectures
All future lecture are tentative.
ISL is the most recommended reading when available.
| Lecture Number |
Date |
Topic |
Materials |
Readings |
| 1 |
9/8/2026 |
|
|
|
| 2 |
9/15/2026 |
|
|
|
| 3 |
9/22/2026 |
|
|
|
| 4 |
9/29/2026 |
|
|
|
| 5 |
10/6/2026 |
|
|
|
| 6 |
10/13/2026 |
|
|
|
| 7 |
10/20/2026 |
Midterm Exam |
|
|
| 8 |
10/27/2026 |
|
|
|
| 9 |
11/3/2026 |
|
|
|
| 10 |
11/10/2026 |
|
|
|
| 11 |
11/17/2026 |
|
|
|
| 12 |
11/24/2026 |
|
|
|
| 13 |
12/1/2026 |
|
|
|
| 14 |
12/8/2026 |
|
|
|
Assignments
There will be four assignments, uniformly weighted. Each assignment is worth 2.5% of the final grade.
Project
You need to conduct a research project, which could be an attempt to beat the state-of-the-art performance on an interesting dataset, or an unexpected application of machine learning algorithms to a different field, or designing a novel algorithm to address a need in machine learning, or theoretically analyzing the performance of a machine learning algorithm (new or old).
A literature survey on a new hot topic in ML is also possible.
You may work individually or in pairs.
All parts of the project are submitted by email to the instructor.
Your project should
- relate to machine learning (obviously)
- allow you to learn something new and interesting
- be significant and publishable in a top machine learning conference
The project proposal will be due on TBD, and is worth 5% of the final grade.
Please concisely describe what your project is about, some related works (no need to be thorough yet), what is your execution plan, what do you expect to learn/contribute, and how are you going to evaluate your results.
We expect the proposal to be 2 to 4 pages (excluding references).
An ungraded update is due on TBD.
This should describe what you've done so far, and what is left to be done.
You are expected to have measurable "work done" by this point.
The project report will be due on TBD, and is worth 25% of the final grade.
Please summarize all your findings (empirical, algorithmic, theoretical) in a scientific report.
We expect there is an introduction section, a background section, a main result section, and a conclusion section.
Depending on your project, you may include an experimental section and/or discussion section.
Please always give proper citations to prior work or results.
Be precise and concise. We expect the report to be at most 8 pages (excluding references).
Your project report will be evaluated on its clarity, significance, rigor, presentation, and completeness.
Books and References
Readings for each lecture will be linked.
Additionally, the following are excellent resources.
- Dive into Deep Learning (D2L), by Aston Zhang, Zack C. Lipton, Mu Li and Alex J. Smola.
- Deep Learning (DL), by Ian Goodfellow, Yoshua Bengio and Aaron Courville.
- Understanding Machine Learning: From Theory to Algorithms (UML), by Shai Shalev-Shwartz and Shai Ben-David..
- Elements of Statistical Learning (ESL), by Trevor Hastie, Robert Tibshirani and Jerome Friedman.
- An Introduction to Statistical Learning (ISL), by Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani.
- Mathematics for Machine Learning (MML), by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong.
If NumPy is new to you, you might consult these resources.
Grading
Grades for this course will be determined with the following breakdown.
- Assignments: 10%.
- Midterm: 30%.
- Final: 30%.
- Project: 30%.
Resources
Here are some top conferences and journals in machine learning. They may be good places to look for project ideas.
I believe all of them should be open access. More generally, you should never have to pay to access an academic article, especially when you are affiliated with a university.
Contact me if you run into any issues accessing any academic article and I'll write up some directions.