Lectures 2019
- Lecture September 24, 2019 - (by Marco Boresta) Questionnaire for checking background (here the results). Description of the exams and phyton background.
- Lecture September 25, 2019 - Introduction to the course. The learning process. Supervised and unsupervised paradigms. Data (trainig testing and validation) (Ref. slide 1_lecture, Bishop – Pattern Recognition and Machine Learning, Springer, 2006 (Chap 1), V. Cherlassky, F. Mulier - Learning from Data, John Wiley and Sons, 2007 (chap.1 and 2) , Bottou, Curtis, and Nocedal - Optimization Methods for LargeScale Machine Learning (section 1 and 2), Teaching Notes chapt. 1)
- Lecture October 1, 2019 - Minimization of the expected risk. Over and under fitting. Hint on Vapnik Chervonenkis theory and Structural risk minimization principle. VC dimension of oriented hyperplanes (The theorem - NO proof) . Hyperplanes with margin - (Ref. slide_2_lecture, Teaching Notes chapt. 1, P. Domingos - A few useful things to know about Machine Learning)
- Lecture October 2, 2019 - The formal neuron and the percepron algorithm (theorem on finite convergence with proof). The voting and the average perceptron Example: the logical OR. The voting and average perceptron. (Ref. slide_3_lecture, Teaching Notes chap. 2)
- Lecture October 8, 2019 - Beyond perceptron. Feedforward Neural Network: Multiplayer perceptron (Ref. Slide 4th Lecture)
- Lecture October 9, 2019 - Hyperparameters and weights problems (Ref. Slide 6th Lecture, [L. Palagi, 2019 - Global optimization issues in deep network regression: an overview] -Section 4) . Python (Numpy) by Marco Boresta (Ref. Homework 1)
- Lecture October 15, 2019 - Short review of optimization: basic definitions and line search algorithms. (Ref. D. Bertsekas, Nonlinear Programming- 2nd ed., L. Grippo and M.- Sciandrone, Metodi di ottimizzazione non vincolata)
- Lecture October 16, 2019 - Feedforward Neural Network: Multiplayer perceptron: shallow netowrk (Ref. Slide 8th Lecture, , Teaching notes chapt. 3). Python (Numpy) by Marco Boresta (Ref. Homework 2 - For advanced topic: question 3 of Perceptron Project or the Rosembrock's function)
- Lecture October 22, 2019 - The output of a Deep Betwrk: forward propagation (Ref. Slide 9th Lecture, Teaching notes chapt. 3)
- Lecture October 23, 2019 - Evaluation test in the class. Basics of gradient batch methods (direction, line searches) (Ref. Slide 10th lecture)
- Lecture October 29, 2019 - Back propagation procedure. Convergence of the batch gradient method with constant stepsize (learning rule) (Ref. Slide 11th lecture)
- Lecture October 30, 2019 - Seminar by Ruggiero Seccia on IBM Watson. Short review of Phyton routines for optimization (Ref. Slide 12th lecture - Phyton). Beyond Batch gradient: samplewise decomposition (Ref. Slide 12th lecture - Methods. In depth reading: Optimization methods for large-scale machine learning by L. Bottou, FE Curtis, J Nocedal). Early stopping rules (Ref. Slide 12th lecture - Early stopping- In depth reading: Early stopping-but when? by Prechelt, Lutz)
- Lecture November 5, 2019 - Beyond Batch gradient: - Decomposition methods for MLP: Extreme learning (Ref. Slide of the 13th lecture, teaching notes chapt. 3)
- Lecture November 6, 2019 - Decomposition methods for MLP (Ref. Slide of the 14th lecture)
- Lecture November 12, 2019 - Radial Basis Function Networks: regularized and generalized RBF network. The XOR example (Ref: Slide of the 15th lecture; Girosi, F. and Poggio, T., 1990. Networks and the best approximation property. Biological cybernetics, 63(3), pp.169-176; Teaching notes chapt. 4)
- Lecture November 13, 2019 - Unsupervised selection of centers. Supervised selection of the centers: Full optimization. Two block decompostion methods, exact and inexact solution of subproblems, convergence properties. Decomposition methods: block learning of centers (Ref. Slide of the 16th lecture, Teaching notes chapt. 4). Discussion on the project 1 with Marco Boresta.
- MIDTERM November 19, 2019
- Lecture November 20, 2019 - Beyond vanilla gradient: monentum term, averaging itateraion, diagonal scaling (Ref. Slide of the 18th lecture). Seminar on Reinforcment Learning by Tommaso Colombo
- Lecture November 26, 2019 - Hard SVM: generalities, defintion of margin, defintion of the max margin problem (Ref. Slide 19th lecture). The primal hard SVM problem
- Lecture November 27, 2019 - Soft SVM: generalities, the primal soft SVM problem. Convex optimization: KKT conditions for linearly constrained porblems. Feasible and descent directions. Frank-wolfe conditional gradient
- Lecture November 27, 2019 - S
- Lecture December 3, 2019 - Quadratic constrained optimzation in Phyton (Marco Boresta). Multiclass classification problems: One against one and one against all
- Lecture December 4, 2019 - Duality in convex quadratic programming. The weak duality theorem (with proof). The Wolfe dual. Construction of the dual Hard and Soft SVM problem.
- Lecture December 10, 2019 - The KKT conditions for the dual SVM problem (eliminating the multipliers).
- Lecture December 11, 2019 - Decomposition algorithms for SVM: the case q=2 (Analytic solution of QP in two dimension) and q>2. Defintion of kernels and examples on the LIBSVM (REF: Slide of the lectures on kernels and LIBSVM graphical interface).
- Lecture December 12, 2019 - (room B2 -11:00-12:30) Exercise on SVM
- Lecture December 17, 2019 - Final term
- Lecture December 18, 2019 - Debriefing on the project