Recommender Systems
statistical methods for machine learning
6 CFU, Laurea magistrale in Informatica (F94)
machine learning
6 CFU, MSc in Data Science and Economics

INSTRUCTOR/DOCENTE: Nicolò Cesa-Bianchi
TAs: Giulia Clerici and Andrea Paudice

News

For students of the MSc in Data Science and Economics

Bibliographic references:

Lecture notes provided by the instructor

The course makes heavy use of probability and statistics. A good textbook on these topics is:

Dimitri P. Bertsekas and John N. Tsitsiklis, Introduction to Probability (2nd edition). Athena Scientific, 2008.

Some good machine learning textbooks:
Shai Shalev-Shwartz e Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.

Mehryar Mohri, Afshin Rostamizadeh e Ameet Talwalkar, Foundations of Machine Learning, MIT Press, 2012.

L. Devroye, L. Gyorfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer, 1996.

Goals

Machine learning is concerned with the design of algorithms that can predict the evolution of a phenomenon based of a set of observations. A standard tool in the development of intelligent systems, machine learning has been successfully applied to a wide range of domains, including vision, speech and language, human-computer interaction, personalized recommendations, health and medicine, autonomous navigation, and many more. The course will describe and analyze, in a rigorous statistical framework, some of the most important machine learning techniques. This will provide the student with a rich set of methodological tools for understanding the general phenomenon of learning in machines.
Note: This is a course about the theoretical foundations of machine learning and the analysis of machine learning algorithms. The focus is on understanding the mathematical principles at the basis of machine learning.

Syllabus

  1. Introduction (version of March 12, 2022)
  2. The Nearest Neighbour algorithm (version of March 12, 2022)
  3. Tree predictors (version of March 17, 2022)
  4. Statistical learning (version of June 1, 2022. Small typo fixed.)
  5. Hyperparameter tuning and risk estimates (version of March 31, 2022)
  6. Risk analysis for tree predictors (version of April 9, 2022)
  7. Risk analysis for Nearest Neighbour (version of April 9, 2022)
  8. Consistency and nonparametric algorithms (version of April 29, 2022. Added a picture.)
  9. Linear prediction (version of April 23, 2022)
  10. Online learning (version of April 29, 2021)
  11. Kernel functions (version of May 9, 2022)
  12. Support Vector Machines (version of June 10, 2020)
  13. Stability and risk control for SVM (version of June 1, 2022)
  14. Boosting and ensemble methods (version of June 1, 2022)
  15. Neural networks and deep learning (version of June 3, 2022)
  16. Logistic regression and surrogate loss functions (version of June 11, 2022)

Experimental projects for students who attended the course in the academic year 2021-22 or earlier:

Warning: The current list of projects is valid until the end of May 2023. This is the only deadline. Projects turned in after May 31, 2023 will be ignored

Exams

The exam consists in writing a paper of about 10-15 pages containing either a report describing experimental results (experimental project) or a in-depth analysis of a theoretical topic (theory project).

For experimental projects:

  1. Fill out this form to choose a project
  2. Create a public repository containing both code and report (in pdf)
  3. Fill out this form to turn in the project
  4. Wait until the project is evaluated (about a week)
  5. Once the project is evaluated, a date for the oral exam can be set.
For theory projects: send an email directly to the instructor.

Oral exam: after the project is evaluated, all students must take an oral exam where they will be asked detailed questions (mathematical definitions, technical proofs, description of algorithms) on the following contents.

IMPORTANT

The experimental project is typically based on implementing two or more learning algorithms (or variants of the same algorithm) from scratch. The algorithms are compared on real-world datasets. The programming language is immaterial. However, the implementation should be reasonable in terms of running time and memory footprint. If the experimental project is based on neural networks, then the student is allowed to use a toolbox (e.g., Tensorflow). The report, preferably written using LaTeX, will be evaluated according to the following criteria:

It is OK if two students submit an experimental project together as a group. Clearly, such group projects are expected to be more extensive than single-person projects (more experiments, comparison with baseline algorithms, etc).

The theory project is typically (but not exclusively) focused on a topic taught in class. The report will be based on one scientific paper (provided by the instructor), and must contain the complete proof of at least a technical result, including all necessary definitions and auxiliary lemmas. The goal of the theory project is to provide an in-depth presentation of the paper's results, including its connections with the related literature. The report may be structured as follows

Note: The theory project report MUST be written in LaTeX.

Course calendar:

Browse the calendar pages and click on a day to find out what was covered on that day, and the link to the video in case the class was viderecorded.