Lectures 2019

 

  1. Lecture September 24, 2019 - (by Marco Boresta) Questionnaire for checking background (here the results). Description of the exams and phyton background.
  2. Lecture September 25, 2019 - Introduction to the course. The learning process. Supervised and unsupervised paradigms. Data (trainig testing and validation) (Ref. slide 1_lecture, Bishop – Pattern Recognition and Machine Learning, Springer, 2006 (Chap 1), V. Cherlassky, F. Mulier - Learning from Data, John Wiley and Sons, 2007 (chap.1 and 2) ,   Bottou, Curtis, and Nocedal - Optimization Methods for LargeScale Machine Learning (section 1 and 2), Teaching Notes chapt. 1)
  3. Lecture October 1, 2019 - Minimization of the expected risk. Over and under fitting.   Hint on Vapnik Chervonenkis theory and Structural risk minimization principle.  VC dimension of oriented hyperplanes (The theorem - NO proof) . Hyperplanes with margin - (Ref. slide_2_lecture, Teaching Notes chapt. 1, P. Domingos - A few useful things to know about Machine Learning)
  4. Lecture October 2, 2019 - The formal neuron and the percepron algorithm (theorem on finite convergence with proof). The voting and the average perceptron Example: the logical OR. The voting and average perceptron. (Ref. slide_3_lecture, Teaching Notes chap. 2)
  5. Lecture October 8, 2019 - Beyond perceptron. Feedforward Neural Network: Multiplayer perceptron (Ref. Slide 4th Lecture)
  6. Lecture October 9, 2019 - Hyperparameters and weights problems (Ref. Slide 6th Lecture, [L. Palagi, 2019 - Global optimization issues in deep network regression: an overview] -Section 4) . Python (Numpy) by Marco Boresta (Ref. Homework 1)
  7. Lecture October 15, 2019 - Short review of optimization: basic definitions and line search algorithms. (Ref. D. Bertsekas, Nonlinear Programming- 2nd ed., L. Grippo and M.- Sciandrone, Metodi di ottimizzazione non vincolata)
  8. Lecture October 16, 2019 - Feedforward Neural Network: Multiplayer perceptron: shallow netowrk (Ref. Slide 8th Lecture, , Teaching notes chapt. 3). Python (Numpy) by Marco Boresta (Ref. Homework 2 - For advanced topic: question 3 of Perceptron Project or the Rosembrock's function)
  9. Lecture October 22, 2019 - The output of a Deep Betwrk: forward propagation (Ref. Slide 9th Lecture, Teaching notes chapt. 3)
  10. Lecture October 23, 2019 - Evaluation test in the class. Basics of gradient batch methods (direction, line searches) (Ref. Slide 10th lecture)
  11. Lecture October 29, 2019 - Back propagation procedure. Convergence of the batch gradient method with constant stepsize (learning rule) (Ref. Slide 11th lecture)
  12. Lecture October 30, 2019 - Seminar by Ruggiero Seccia on IBM Watson. Short review of Phyton routines for optimization (Ref. Slide 12th lecture - Phyton). Beyond Batch gradient: samplewise decomposition (Ref. Slide 12th lecture - Methods. In depth reading:  Optimization methods for large-scale machine learning by L. Bottou, FE Curtis, J Nocedal). Early stopping rules (Ref. Slide 12th lecture - Early stopping- In depth reading: Early stopping-but when? by  Prechelt, Lutz)
  13. Lecture November 5, 2019 - Beyond Batch gradient: - Decomposition methods for MLP: Extreme learning (Ref. Slide of the 13th lecture, teaching notes chapt. 3)
  14. Lecture November 6, 2019 -  Decomposition methods for MLP (Ref. Slide of the 14th lecture)
  15. Lecture November 12, 2019 - Radial Basis Function Networks: regularized and generalized RBF network. The XOR example (Ref: Slide of the 15th lecture; Girosi, F. and Poggio, T., 1990. Networks and the best approximation property. Biological cybernetics, 63(3), pp.169-176; Teaching notes chapt. 4)
  16. Lecture November 13, 2019 - Unsupervised selection of centers. Supervised selection of the centers: Full optimization. Two block decompostion methods, exact and inexact solution of subproblems, convergence properties. Decomposition methods: block learning of centers (Ref. Slide of the 16th lecture, Teaching notes chapt. 4). Discussion on the project 1 with Marco Boresta.
  17. MIDTERM November 19, 2019
  18. Lecture November 20, 2019 - Beyond vanilla gradient: monentum term, averaging itateraion, diagonal scaling (Ref. Slide of the 18th lecture). Seminar on  Reinforcment Learning by Tommaso Colombo
  19. Lecture November 26, 2019 - Hard SVM: generalities, defintion of margin, defintion of the max margin problem (Ref. Slide 19th lecture). The primal hard SVM problem
  20. Lecture November 27, 2019 -  Soft SVM: generalities, the primal soft SVM problem. Convex optimization: KKT conditions for linearly constrained porblems. Feasible and descent directions. Frank-wolfe conditional gradient
  21. Lecture November 27, 2019 -  S
  22. Lecture December 3, 2019 -  Quadratic constrained optimzation in Phyton (Marco Boresta). Multiclass classification problems: One against one and one against all
  23. Lecture December 4, 2019 -  Duality in convex quadratic programming. The weak duality theorem (with proof). The Wolfe dual. Construction of the dual Hard and Soft SVM problem.
  24. Lecture December 10, 2019 -   The KKT conditions for the dual SVM problem (eliminating the multipliers).
  25. Lecture December 11, 2019 -  Decomposition algorithms for SVM: the case q=2 (Analytic solution of QP in two dimension) and q>2. Defintion of kernels and examples on the LIBSVM (REF: Slide of the lectures on kernels and LIBSVM graphical interface). 
  26. Lecture December 12, 2019 - (room B2 -11:00-12:30) Exercise on SVM
  27. Lecture December 17, 2019 - Final term
  28. Lecture December 18, 2019 - Debriefing on the project