Technische Universität München, Department of Mathematics
Lecture
Optimization Methods for Machine Learning (Modern Methods in Nonlinear Optimization)
Summer Term 2017
Contents
Machine learning has become a highly important field of research, especially in the context of big data. Many models in supervised learning such as support vector machines or neural networks require training based on data, which calls for suitable nonlinear optimization techniques.
This course gives an introduction to modern optimization methods that are well-suited for machine learning tasks. In particular, they
a) take into account the specific problem structure that arises in empirical risk minimization, b) are compatible with the results of statistical learning theory, and c) are designed to handle huge amounts of data efficiently. Numerical aspects and illustrative examples will also be part of the lecture.
Module number: MA4503. The module description can be found
here.
Prerequisites:
MA2503 (Nichtlineare Optimierung: Grundlagen),
MA3503 (Nonlinear Optimization: Advanced)
News
Dates
Lecture Notes
A
current version of the lecture notes is available for download (last update: July 26th, 2017).
The lecture notes will be dynamically updated as the course proceeds.
Exercises
- Sebastian Garreis and Philipp Jarde are responsible for the exercises. In case of questions or suggestions, please feel encouraged to contact garreis
ma.tum.de or jardema.tum.de.
- The exercise classes take place biweekly.
- For participation in the exercise classes registration on TUMonline is required.
- Please take a look at the exercise sheet and try to solve the problems before the respective exercise class. The content of the tutorials will be based on your questions.
- Solution sketches of the problems that could not be discussed in at least one of the classes will be provided here. We encourage you to solve the remaining problems on your own before taking a look at the solution.
Exam
| Dates |
| Main exam |
Friday, August 4th, 2017 |
14:00 - 15:00 |
MW 0001 |
| Resit exam |
Friday, October 6th, 2017 |
11:00 - 12:00 |
MW 1801 |
Important information (about the main exam):
- Registration via TUMonline is mandatory for participation in the exam (registration for the lecture and/or the exercise classes is not sufficient)! Registration for the main exam starts on Monday, May 29th, 2017, and ends on Friday, June 30th, 2017. Please do not forget to register until that date - you will not be allowed to take the exam without prior registration on TUMonline.
- In case you have been granted any special regulations for your examination, please make sure to inform us (garreisma.tum.de) right after registration and not later than July 21st, 2017. Failure to notify us by that time means you voluntarily forfeit your right to any special regulations for that exam.
- The dates, times and places regarding the exam and the registration, which are posted above, are preliminary and subject to change. Please consult TUMonline for the official dates.
- All topics covered in the lecture, in the exercise classes or on the exercise sheets are relevant for the exam.
- Each student is allowed to bring one self-made and handwritten sheet of A4 paper (with arbitrary notes on both sides of it) for his/her personal use in the exam. Copies and any form of text that is created or generated by software, computer programs or other tools are not allowed.
- The duration of the exam is 60 minutes. Additionally, before the beginning of the examination, each student will be given 5 minutes to read the exam paper. In this period, writing is not allowed.
- Please make sure to be in the examination room at least 10-15 minutes prior to the scheduled starting time.
- Bring a photo ID (passport or identity card) and your student ID. We will check the IDs during the exam.
- On the doors of the examination room a list of matriculation numbers and seat numbers will be posted. Please find your matriculation number and locate the correct seat in the examination room. Please keep the empty rows free of luggage and other obstacles.
- Be sure to switch off any mobile phones, calculators, tablet computers, smart watches and other electronic gear and store it out of sight in your bags. Handling any kind of electronic device, whether switched on or not, will be considered an attempt at cheating.
- Please do not use red or green pens nor a pencil.
Inspection (Klausureinsicht):
- The exam inspection takes place on Monday, August 21st, 2017, from 2:00 pm to 3:00 pm in 03.08.011.
- Please bring a photo ID (the student ID is not sufficient).
- If you cannot come yourself, you can authorize a representative to inspect the exam for you and (if necessary) demand a revision of the grading on your behalf. To do this, please issue a written authorization, which explicitly states your name and date of birth and the name and date of birth of the authorized person, and sign it. The authorized person will have to present a photo ID.
- Applications for revised grading (Zweitkorrektur) are only possible during the inspection.
Information about the resit exam:
- Registration via TUMonline is mandatory for participation in the resit exam (registration for the lecture and/or the exercise classes is not sufficient)! Registration for the resit exam starts on Monday, September 11th, 2017, and ends on Monday, September 25th, 2017. Please do not forget to register until that date - you will not be allowed to take the exam without prior registration on TUMonline.
- In case you have been granted any special regulations for your examination, please make sure to inform us (garreisma.tum.de) right after registration and not later than October 2nd, 2017. Failure to notify us by that time means you voluntarily forfeit your right to any special regulations for that exam.
- The dates, times and places regarding the resit exam and the registration, which are posted above, are preliminary and subject to change. Please consult TUMonline for the official dates.
- The exam inspection takes place on Tuesday, Octover 17th, 2017, from 9:30 am to 10:30 am in 03.08.035.
- The rest of the regulations for the resit exam does not differ from the ones for the main exam (see above).
Literature
This list will be updated as the course proceeds.
- L. Bottou, F. E. Curtis, J. Nocedal: Optimization methods for large-scale machine learning
, Technical Report 1606.04838, arXiv, 2016
- T. Glasmachers, C. Igel, Maximum-gain working set selection for SVMs, Journal of Machine Learning Research 7, 1437–1466, 2006.
- N. List and H. U. Simon: A general convergence theorem for the decomposition method. In: J. Shawe-Taylor and Y. Singer, eds., Proceedings of the 17th Annual Conference on Learning Theory, COLT 2004, LNCS 3120, 363–377, Springer-Verlag, 2004
- V. N. Vapnik: The Nature of Statistical Learning Theory, Springer-Verlag, 1995
- V. N. Vapnik: An overview of statistical learning theory, IEEE Transactions on Neural Networks 10, 988–999, 1999
