Technische Universität München, Zentrum Mathematik
Vorlesung
Modern Methods in Nonlinear Optimization - Optimization in Machine Learning
Summer term 2021
Course Content
Machine learning has become a highly important field of research, especially in the context of big data. Many models in supervised learning such as support vector machines or neural networks require training based on data, which calls for suitable nonlinear optimization techniques. This course gives an introduction to modern optimization methods that are well-suited for machine learning tasks. In particular, they a) take into account the specific problem structure that arises in empirical risk minimization, b) are compatible with the results of statistical learning theory, and c) are designed to handle huge amounts of data efficiently. Numerical aspects and illustrative examples will also be part of the lecture.
A table of contents of the lecture:
- Introduction and Problem Setting of Supervised Learning
- Some Aspects of Statistical Learning Theory
- Support Vector Machines (SVM) and Decomposition Methods (SMO)
- Stochastic (Sub)gradient Methods (SG)
- (Deep) Neural Networks
- Extensions of SG: Variance Reduction, Nonconvexity, Proximal Methods
News
| 01.04.2021 |
Welcome to the course Modern Methods in Nonlinear Optimization - Optimization in Machine Learning. Detailled information on the lecture and exercises will be posted as well in the Moodle course. |
People
- Lecture
- Exercise coordination
Lecture
The lecture is held on
Thursdays, 10:15-11:45 on Zoom. The Zoom link can be found on the Moodle page.
Literature
- L. Bottou, F. E. Curtis, J. Nocedal: Optimization methods for large-scale machine learning
, SIAM Review 60, 2, 223–311, 2018.
- T. Glasmachers, C. Igel, Maximum-gain working set selection for SVMs, Journal of Machine Learning Research 7, 1437–1466, 2006.
- V. N. Vapnik: The Nature of Statistical Learning Theory, Springer-Verlag, 1995.
- V. N. Vapnik: An overview of statistical learning theory, IEEE Transactions on Neural Networks 10, 988–999, 1999.
- S. Sra, S. Nowozin, S. J. Wright, eds., Optimization for Machine Learning, The MIT Press, Cambridge, 2012.
