statistical methods for machine learning 6 CFU, MSc in Computer Science machine learning and statistical learning 6 CFU, MSc in Data Science for Economics 2025-26 edition INSTRUCTOR/DOCENTE: Nicolò Cesa-Bianchi TAs: Emmanuel Esposito and Luigi Foscari

News

• Note: The written test on March 26 starts at 14:30.

• A slighty revised version of the quiz list is available.

• The experimental projects for this edition of the course are here.

• To take the written test, you must register to the corresponding exam session. For DSE/DSEH students, the exam session appears under the name MACHINE LEARNING AND STATISTICAL LEARNING (B7903-).

• For the past edition of this course, follow this link.

Course stats

• Course satisfaction (out of 10)
  2024-25: 8.55 (CS), 8.28 (DSE).
  2023-24: 7.69 (CS), 8.07 (DSE).
  2022-23: 8.07 (CS), 7.60 (DSE).

• Average grade (out of 30)
  2024-25: 26.8 (CS), 25.0 (DSE, ML module only), 30.0 (MAT), 21.5 (ERASMUS).
  2023-24: 27.7 (CS), 25.3 (DSE, ML module only), 28.0 (MAT), 24.3 (ERASMUS).
  2022-23: 27.2 (CS), 26.2 (DSE, ML module only), 29.0 (MAT), 25.1 (ERASMUS).

For students of the MSc in Data Science for Economics

The course Machine learning and statistical learning has two separate exams, one for the MACHINE LEARNING module (Cesa-Bianchi, 40 hours, this course) and one for the STATISTICAL LEARNING module (Salini, 40 hours). The exams for the two modules can be taken in different exam sessions.

The written test for the MACHINE LEARNING module will be offered on dates that do not necessarily coincide with the DSE exam sessions. These dates are shown in the calendar entries below here.

Prerequisites:

• Calculus: Taylor's formula, multivariate functions, gradient.
• Probability and statistics: discrete probability, conditional probability, random variable, expectation, conditional expectation, independence, random sample, sample mean, law of large numbers.
• Linear algebra: inner product spaces, matrix calculus.

Bibliographic references:

Lecture notes provided by the instructor

A good textbook on probability and statistics is:

Dimitri P. Bertsekas and John N. Tsitsiklis, Introduction to Probability (2nd edition). Athena Scientific, 2008.

Shai Shalev-Shwartz e Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.

Mehryar Mohri, Afshin Rostamizadeh e Ameet Talwalkar, Foundations of Machine Learning (2nd edition), MIT Press, 2018.

Goals

Machine learning is the core technology driving the recent revolutionary advances in Artificial Intelligence (AI). While much of this extraordinary progress relies on the paradigm of Generative AI (GenAI), building large-scale systems—such as ChatGPT or Gemini—requires the entire toolbox of ML techniques. Therefore, to comprehend modern AI, we must first understand the tools and principles that underpin its foundations.
This course is concerned with the statistical and algorithmic foundations of supervised machine learning. Emphasis is on theory and principled methods as opposed to practice and heuristics. The course is propedeutic to advanced courses in deep learning, reinforcement learning, and generative AI.

Syllabus

1. Introduction (version of January 13, 2026)
2. The Nearest Neighbour algorithm (version of January 13, 2026)
3. Tree predictors (version of January 18, 2026)
4. Statistical learning (version of January 27, 2026)
5. Risk analysis for tree predictors (version of February 1, 2026)
6. Hyperparameter tuning and risk estimates (version of August 28, 2025).
7. Consistency and nonparametric algorithms (version of February 2, 2026).
8. Risk analysis for Nearest Neighbour (version of February 3, 2026)
9. Linear predictors (version of February 10, 2026)
10. Online gradient descent (version of February 24, 2026; thanks to Marco Baderna for typos!)
11. Kernel functions (version of March 8, 2026)
12. Support Vector Machines (March 8, 2026)
13. Stability and risk control for SVM (version of May 26, 2023)
14. Neural networks and deep learning (version of March 8, 2026)
15. Logistic regression and surrogate loss functions (version of June 8, 2023)
16. Boosting and ensemble methods (version of June 1, 2024). This handout has not been covered in class and is optional.

Some notebooks with small experiments showing overfitting and hyperparameter tuning.

The exam consists of two parts:

1. Writing a paper containing either a report describing experimental results (experimental project) or a in-depth analysis of a theoretical topic (theory project).
2. Taking a written test on all the topics covered in class. The test consists in a list of 12 questions taken from this list (updated from time to time), plus an extra bonus question. During the test, it is forbidden to access any material (in paper or digital format) besides the test content.

The written test can be only taken at the regular exam sessions. Written test and experimental projects can be passed in any order. The final grade is a weighted average (rounded to the nearest integer) of the mark obtained in the written test (60%) and the mark obtained in the project (40%). The exam is passed if: the average is 18 or higher and both marks are 17 or higher.

The experimental projects for this edition of the course are here. For any matters related to the experimental projects please contact the TAs: emmanuel.esposito@unimi.it and luigi.foscari@unimi.it; messages sent to the instructor will be ignored.

The theory project is typically (but not exclusively) focused on a topic taught in class. The report (typically 10-15 pages) will be based on one scientific paper (provided by the instructor), and must contain the complete proof of at least a technical result, including all necessary definitions and auxiliary lemmas. The goal of the theory project is to provide an in-depth presentation of the paper's results, including its connections with the related literature. The report may be structured as follows

• Introduction and description of the problem
• Most important related works
• Notation and relevant definitions
• Proof of a technical result
• Some critical considerations.

Steps to complete the theory project:

1. Send an email directly to the instructor to agree on a topic (typically, but not exclusively chosen among those covered in class);
2. The instructor will then suggest one or two papers in that area;
3. Turn in the report via email to the instructor;
4. After 1-2 weeks, the instructor will get in touch to agree on a date for the oral discussion.

IMPORTANT FOR ALL PROJECTS: Your report must contain the following declaration: I/We declare that this material, which I/We now submit for assessment, is entirely my/our own work and has not been taken from the work of others, save and to the extent that such work has been cited and acknowledged within the text of my/our work. I/We understand that plagiarism, collusion, and copying are grave and serious offences in the university and accept the penalties that would be imposed should I engage in plagiarism, collusion or copying. This assignment, or any part of it, has not been previously submitted by me/us or any other person for assessment on this or any other course of study.

Course calendar:

Browse the calendar pages and click on a day to find out what was covered on that day.