Cours : MAC0460/5832 - Introdução ao aprendizado de máquina

Généralités

Tout replier Tout déplier

Encontros ONLINE : https://meet.google.com/whv-mdvw-hnt

(a partir de 30/03, nos horários de aula)

Avaliação

Critério de avaliação: inclui as seguintes atividades

Lista1
5 EPs
Questionário de avaliação final (google form)

Média final: MF = [(ML + 3*MEP)/4]*alpha

ML = Lista1

MEP = (1.0*EP1 + 2.0*EP2 + 2.5*EP3 + 2.5*EP4 + 2.0*EP5)/10

alpha em [0, 1.1] ~ Coeficiente baseado no Questionário de avaliação final

No caso da pós a MF será convertida para conceito.

Prazo final para entregas: 27/julho

(para a graduação, o prazo preliminarmente estabelecido para a REC é 15/agosto)

Reference material

Main reference: CALTECH Machine Learning Course (https://work.caltech.edu/telecourse.html). Book "Learning from Data" by Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin

Other references: some topic specific references as well as slides with some notes are indicated along the class log below.

Practice:

Python: https://python.ime.usp.br/panda/python
Notebook with a very quick intro to some useful NumPy libraries (notebook used in the Curso de Verão 2020): aqui
Numpy, Python notebooks: http://cs231n.github.io/
scikit-learn: https://scikit-learn.org/stable/
Google Colab: https://colab.research.google.com/

Tasks

All practical and written exercises will be posted here.

Class log

Marqué

02.03 - Introdução. IA x ML x Data Science. ML consiste de ferramentas. ML (supervisionado) é um modelo geral de processamento de dados, cujo ajuste de parâmetros é feito usando-se exemplos, guiado por uma função de custo a ser otimizada. ML x Statistics, Teorema de Bayes, abordagem discriminativa x gerativa.

05.03 - Perceptron algorithm (slides 10 to 13 of Lecture 1; Section "1.1.2 A simple learning model" of the text book ); quick intro to linear regression (Section 3.2 of the text book) There is a proof of the convergence of the perceptron algorithm in Section 9.1.2 of https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf Enjoy!

Slides with some notes: perceptron.pdf , linear_regression.pdf )

09.03 - The gradient descent algorithm (idea, derivation of weight update formula for the MSE cost function) + more binary classification (cases when the data is not linearly separable, pocket algorithm, use of linear regression for binary classification) and quick intro to logistic regression (most of these are in Chapter 3 of the text book .

Slides : gradient_descent.pdf

12.03 - Jubileu de ouro do IME (não haverá aula) / 50th anniversary of the institute (no class)

~~Classes are SUSPENDED from 16/03 on. We are evaluating ways to proceed classes remotely using some online platform. Stay tuned for AVISOS.~~

16.03 - ~~Plan: logistic regression (formulation of the problem, cross-entropy loss, weight update rule)~~

19.03 - Tentative online meeting. I think it worked ok.

We will start online meetings to discuss ML contents from 30.03 on

(if you are unable to attend due to lack of access to internet or of suitable equipment, let me know)

30.03 Logistic regression + gradient descent

in-house Slides
Slides CALTECH: Slides 09, pages 9-24
Video CALTECH: Lecture 09 (24min~end)

Vídeo do meeting: https://drive.google.com/open?id=1tPT2cW10P-Lv5qok6N-fvHHVcHBg2rWG

02.04 Multiclass classification

in-house slides multiclass.pdf (updated: 05/04)
Multinomial logistic regression (wikipedia)

Video do encontro: https://drive.google.com/file/d/1LoYTJkW5uoUVuNTb7U_8flcMWbimh70i/view?usp=sharing

06/04 Overfitting + performance evaluation

Overfitting, E_in, E_out, E_val : overfitting.pdf
Evaluation of classifiers: evaluation.pdf (up to page 11)

Video of the meeting: https://drive.google.com/file/d/1gINItriZErKfVm7qQwMHzjNSx7p9fVkQ/view?usp=sharing

(In the book, overfitting is treated in Chapter 4. It gives an explanation making references to theoretical formulations presented in the previous chapters. It also presents regularization and validation as means of dealing with overfitting. For now, we will just present a brief introduction to overfitting and how to evaluate the performance of the classifiers. Later, after studying the theoretical aspects of learning, we will come back to these topics)

09/04 - Performance measure computation + model selection

Topics discussed

we have seen some performance metrics; we discussed how to compute (estimate) those metrics
among the techniques, one is cross-validation: Section 4.3 of the reference book
model selection: how cross-validation can be used to select a model
slides: evaluation.pdf

Video of the meeting: https://drive.google.com/file/d/13yYwo03Xn1RX1rd649w4ZsZHebKf3r0P/view?usp=sharing

13/04 - Review of Logistic Regression

There will be no online meeting this day.

Recommended activity: videos on binary classification / Logistic regression by Andrew Ng :

They are short videos from "Binary Classification (C1W2L01)" to "Explanation of Logistic Regression's Cost Function (C1W2L18)"

You will learn how to implement an efficient code in Python for computing logistic regression. It also should help you to do EP3 and EP4, as well as machine learning practice (especially deep learning).

16/04 - Is learning feasible ?

Let us start discussing the theoretical aspects of machine learning.

We will follow prof. Abu-Mostafa's lectures, starting with Lecture 02 ( Is learning feasible ? )

Video of the meeting: https://drive.google.com/file/d/1AWbpDvFJd9Z_-Zm15Owm1H_q6TvSI2GZ/view?usp=sharing

23/04 - Lecture 05 (Training versus Testing)

Important concepts to be used later:

dichotomies
growth function
break point for a hypothesis set

Lecture 5:

Slides used in the meeting: theory2.pdf

Video of the meeting: https://drive.google.com/file/d/1zjXjYXCIFEBfexRrKwmSM8IphXi7l9tS/view?usp=sharing

27.04 -- Lecture 06 (Theory of Generalization)

Lecture 06:

Section 2.1.2 and 2.1.4 of the book (there is a long proof in the Appendix, not covered in the video)

Slides of the meeting: theory3.pdf

Video of the meeting: https://drive.google.com/file/d/1TLnzlexNCCDdwVXwajTFy7TIek50mT8h/view?usp=sharing

30.04 -- Lecture 07 ( The VC dimension)

Lecture 07 (video) : https://youtu.be/Dc0sr0kdBVI

Sections 2.1 to 2.2.3

Slides of the meeting: theory4.pdf

Video of the presentation: https://drive.google.com/file/d/12qsxvPilJNFoTx3QGGCX0gs3W1wKPziB/view?usp=sharing ( the first 10 minutes of the meeting have not been recorded ...)

07.05 - Lecture 08

We will have no meeting on 04/05 (next Monday), to allow some time for you to catch up with the contents we are discussing (VC theory) and to prepare for the next meeting (Bias-variance model, Lecture 08)

Lecture 08:

Slides of Lecture 08: http://work.caltech.edu/slides/slides08.pdf

Video of the meeting: https://drive.google.com/file/d/1auRe-R8A8kTr3R0gbcklz64USx7D9C7S/view?usp=sharing

18.05 -- Neural networks

No classes on May 11th and 14th (break week)

Next meeting (18/05): we will start discussions on neural networks. We will not follow Lecture 10 of Caltech strictly. Nevertheless, the following references might be useful

Lecture 10: https://youtu.be/Ih5Mr93E-2c (from 20min on)
A nice online book on Neural Networks (for those who are really interested): http://neuralnetworksanddeeplearning.com/ (Neural Networks and Deep Learning, by Michael Nielsen)

Slides: non_linear.pdf neural_nets.pdf (updated on 21/05)

Video of the meeting: https://drive.google.com/file/d/1vrPPYVsIgu-2nA1HRtlWn1Pa_QQo625f/view?usp=sharing