CSCI-GA 3033 - Honors Machine Learning - Fall 2026

Logistics

Course Description

An advanced introduction to machine learning. This course is intended for PhD students who wish to conduct technical research in machine learning. It will consequently have higher expectations in terms of preparation, and aim to leave you with a deeper understanding of the material. It is intended to serve as a foundation for research in machine learning as well as a prerequisite for more advanced machine learning courses.

Topics will include learning fundamentals (loss functions, empirical risk minimization, overfitting, regularization, maximum likelihood estimation, and maximum a posteriori estimation); optimization fundamentals (gradient descent and stochastic gradient descent); classical supervised learning methods (linear regression, k-nearest neighbors, logistic regression, support vector machines, kernels, decision trees, boosting, and bagging); unsupervised learning methods (principal component analysis, k-means clustering, and Gaussian mixture models); neural networks (multilayer perceptrons, deep networks, convolutional neural networks, and deep generative models); and additional topics including reinforcement learning, attention, and transformers.

This is the first time this course is being offered, and thus details are subject to change.

Feedback on this course or the instructor is always welcome, whether in-person, by email, or anonymously at this link.

Prerequisites

Undergraduate courses in linear algebra and strong programming skills for implementation of algorithms studied in class. Strong knowledge of probability.

Recommended: Knowledge of vector calculus, elementary statistics, and MATH-UA 120: Discrete Mathematics.

This course versus CSCI-GA.2565

In Fall 2026, CSCI-GA 2565 - Machine Learning is also offered by the Department of Computer Science. Although there is a significant degree of overlap in the content of these courses, CSCI-GA 3033 is designed to target PhD students in computer science who will specialize and conduct research in machine learning, while CSCI-GA 2565 is designed for general PhD and MSc level students in computer science regardless of their specializations.

Lectures

All future lecture are tentative. ISL is the most recommended reading when available.
Lecture Number Date Topic Materials Readings
1 9/8/2026 Logistics, Classification, k-Nearest Neighbors Lecture 00 (PDF), Lecture 01 (PDF) UML Section 19
ESL Section 2.3.2, 13.3
ISL Section 2.2
2 9/15/2026 Perceptron, Logistic Regression UML Section 9.1, 9.3
ESL Section 4.4, 4.5
ISL Section 4.3
3 9/22/2026 Support Vector Machines
4 9/29/2026 Kernels, Linear Regression
5 10/6/2026 Decision Trees and Aggregation Methods
6 10/13/2026 Principal Component Analysis, k-Means, and Gaussian Mixture Models
7 10/20/2026 Midterm Exam
8 10/27/2026 Neural Networks
9 11/3/2026 Optimization, Convolutional Neural Networks
10 11/10/2026 Deep Generative Models
11 11/17/2026 Attention and Transformers
12 11/24/2026 Reinforcement Learning
13 12/1/2026 Experimental Methodology and Research Practices
14 12/8/2026 [Flex day to fit other material or advanced content]

Assignments

There will be three assignments, uniformly weighted. Each assignment is worth 3.33% of the final grade.

Project

You need to conduct a research project on machine learning. You may work individually or in pairs. All parts of the project are submitted by email to the instructor.

This course is intended to prepare you for Ph.D.-level research in machine learning. Consequently, in terms of both scope and quality, your project should be appropriate for a strong conference in machine learning. Your project should engage with machine learning topics at methodological or low level (i.e., it should do more than "just call APIs"), feel free to ask if you're unsure if something is acceptable. The project proposal will be due on October 5, and is worth 5% of the final grade. Please concisely describe what your project is about, some related works (no need to be thorough yet), what is your execution plan, what do you expect to learn/contribute, and how are you going to evaluate your results. We expect the proposal to be 2 to 4 pages (excluding references).

An ungraded update is due on November 9. This should describe what you've done so far, and what is left to be done. You are expected to have measurable "work done" by this point.

The project report will be due on December 14, and is worth 25% of the final grade. Please summarize all your findings (empirical, algorithmic, theoretical) in a scientific report. We expect there is an introduction section, a background section, a main result section, and a conclusion section. Depending on your project, you may include an experimental section and/or discussion section. Please always give proper citations to prior work or results. Be precise and concise. We expect the report to be at most 8 pages (excluding references).

Your project report will be evaluated on its clarity, significance, rigor, presentation, and completeness.

Books and References

Readings for each lecture will be linked. Additionally, the following are excellent resources. If NumPy is new to you, you might consult these resources.

Grading

Grades for this course will be determined with the following breakdown.

Resources

Here are some top conferences and journals in machine learning. They may be good places to look for project ideas. I believe all of them should be open access. More generally, you should never have to pay to access an academic article, especially when you are affiliated with a university. Contact me if you run into any issues accessing any academic article and I'll write up some directions.