CSCI-GA 3033 - Honors Machine Learning - Fall 2026
Logistics
- Instructor: Gautam Kamath
- Course Communication: Campuswire, enrolled students can ask instructor for code.
- Lecture: Tuesdays, 4:55 PM - 6:55 PM.
- Location: Silv 401.
- Midterm Exam: October 27.
- Final Exam: TBD.
- Office Hours: Wednesdays 12 PM to 1 PM, 60FA Room 510.
- Teaching Assistants: TBD (there should be one).
- Syllabus: here.
Course Description
An advanced introduction to machine learning. This course is intended for PhD students who wish to conduct technical research in machine learning. It will consequently have higher expectations in terms of preparation, and aim to leave you with a deeper understanding of the material. It is intended to serve as a foundation for research in machine learning as well as a prerequisite for more advanced machine learning courses.
Topics will include learning fundamentals (loss functions, empirical risk minimization, overfitting, regularization, maximum likelihood estimation, and maximum a posteriori estimation); optimization fundamentals (gradient descent and stochastic gradient descent); classical supervised learning methods (linear regression, k-nearest neighbors, logistic regression, support vector machines, kernels, decision trees, boosting, and bagging); unsupervised learning methods (principal component analysis, k-means clustering, and Gaussian mixture models); neural networks (multilayer perceptrons, deep networks, convolutional neural networks, and deep generative models); and additional topics including reinforcement learning, attention, and transformers.
This is the first time this course is being offered, and thus details are subject to change.
Feedback on this course or the instructor is always welcome, whether in-person, by email, or anonymously at this link.
Prerequisites
Undergraduate courses in linear algebra and strong programming skills for implementation of algorithms studied in class. Strong knowledge of probability.
Recommended: Knowledge of vector calculus, elementary statistics, and MATH-UA 120: Discrete Mathematics.
This course versus CSCI-GA.2565
In Fall 2026, CSCI-GA 2565 - Machine Learning is also offered by the Department of Computer Science. Although there is a significant degree of overlap in the content of these courses, CSCI-GA 3033 is designed to target PhD students in computer science who will specialize and conduct research in machine learning, while CSCI-GA 2565 is designed for general PhD and MSc level students in computer science regardless of their specializations.
Lectures
All future lecture are tentative.
ISL is the most recommended reading when available.
| Lecture Number |
Date |
Topic |
Slides |
Readings and Other Materials |
| 1 |
9/8/2026 |
Logistics, Classification, k-Nearest Neighbors |
Lecture 00 (PDF), Lecture 01 (PDF) |
UML Section 19, 11.2
ESL Section 2.3.2, 13.3, 7.10
ISL Section 2.2, 5.1
|
| 2 |
9/15/2026 |
Perceptron, Logistic Regression |
Lecture 02 (PDF) |
UML Section 9.1, 9.3
ESL Section 4.4, 4.5.1
ISL Section 4.3
Interactive perceptron demo
Interactive logistic regression demo (work in progress)
|
| 3 |
9/22/2026 |
Logistic Regression Continued, Optimization |
Optimization (PDF) |
Optimization (supplementary):
UML Section 14.1, 14.3
|
| 4 |
9/29/2026 |
Linear Regression, Support Vector Machines |
Linear Regression (PDF), Support Vector Machines (PDF) |
Linear Regression:
UML Section 9.2
ESL Section 3.2, 3.4.1-3.4.3
ISL Section 3.1-3.2, 6.2
Calculus derivation
Support Vector Machines:
UML Section 15, 14.2
ESL Section 4.5.2, 12.1-12.2
ISL Section 9.1-9.2
|
| 5 |
10/6/2026 |
Kernels, Decision Trees |
|
Kernels:
ESL Section 12.3
|
| 6 |
10/13/2026 |
Aggregation Methods |
|
|
| 7 |
10/20/2026 |
Principal Component Analysis, k-Means, and Gaussian Mixture Models |
|
|
| 8 |
10/27/2026 |
Midterm Exam |
|
|
| 9 |
11/3/2026 |
Neural Networks |
|
|
| 10 |
11/10/2026 |
Optimization, Convolutional Neural Networks |
|
|
| 11 |
11/17/2026 |
Deep Generative Models |
|
|
| 12 |
11/24/2026 |
Attention and Transformers |
|
|
| 13 |
12/1/2026 |
Reinforcement Learning |
|
|
| 14 |
12/8/2026 |
Experimental Methodology and Research Practices |
|
|
Assignments
There will be three assignments, uniformly weighted. Each assignment is worth 3.33% of the final grade.
All dates are tentative.
- A1: Assignment 1 (PDF), Assignment files (ZIP). Release date: September 21. Due date: October 12. Possible topics: k-nearest neighbors, perceptron, logistic and linear regression, support vector machines, kernels, and decision trees.
- A2: Release date: October 12. Due date: November 9. Possible topics: aggregation methods, principal component analysis, k-means, Gaussian mixture models, and introductory neural networks.
- A3: Release date: November 9. Due date: December 7. Possible topics: optimization, convolutional neural networks, deep generative models, attention and transformers, and reinforcement learning.
Project
You need to conduct a research project on machine learning.
You may work individually or in pairs.
All parts of the project are submitted by email to the instructor.
This course is intended to prepare you for Ph.D.-level research in machine learning.
Consequently, in terms of both scope and quality, your project should be appropriate for a strong conference in machine learning.
Your project should engage with machine learning topics at methodological or low level (i.e., it should do more than "just call APIs"), feel free to ask if you're unsure if something is acceptable.
The project proposal will be due on October 5, and is worth 5% of the final grade.
Please concisely describe what your project is about, some related works (no need to be thorough yet), what is your execution plan, what do you expect to learn/contribute, and how are you going to evaluate your results.
We expect the proposal to be 2 to 4 pages (excluding references).
An ungraded update is due on November 12.
This should describe what you've done so far, and what is left to be done.
You are expected to have measurable "work done" by this point.
The project report will be due on December 14, and is worth 25% of the final grade.
Please summarize all your findings (empirical, algorithmic, theoretical) in a scientific report.
We expect there is an introduction section, a background section, a main result section, and a conclusion section.
Depending on your project, you may include an experimental section and/or discussion section.
Please always give proper citations to prior work or results.
Be precise and concise. We expect the report to be at most 8 pages (excluding references).
Your project report will be evaluated on its clarity, significance, rigor, presentation, and completeness.
Interactive Demos
Explore the machine learning demos to experiment with the classifiers from class.
Books and References
Readings for each lecture will be linked.
Additionally, the following are excellent resources.
- Dive into Deep Learning (D2L), by Aston Zhang, Zack C. Lipton, Mu Li and Alex J. Smola.
- Deep Learning (DL), by Ian Goodfellow, Yoshua Bengio and Aaron Courville.
- Understanding Machine Learning: From Theory to Algorithms (UML), by Shai Shalev-Shwartz and Shai Ben-David.
- Elements of Statistical Learning (ESL), by Trevor Hastie, Robert Tibshirani and Jerome Friedman.
- An Introduction to Statistical Learning (ISL), by Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani.
- Mathematics for Machine Learning (MML), by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong.
If NumPy is new to you, you might consult these resources.
Grading
Grades for this course will be determined with the following breakdown.
- Assignments: 10%.
- Midterm: 30%.
- Final: 30%.
- Project: 30%.
Resources
Here are some top conferences and journals in machine learning. They may be good places to look for project ideas.
I believe all of them should be open access. More generally, you should never have to pay to access an academic article, especially when you are affiliated with a university.
Contact me if you run into any issues accessing any academic article and I'll write up some directions.