International Journal of Nonlinear Analysis and Applications

Training analysis of optimization models in machine learning

Document Type : Research Paper

Authors

Department of Mathematics Science, Ministry of Education, Babylon, Iraq

Abstract

Machine learning is fast evolving, with numerous theoretical advances and applications in a variety of domains. In reality, most machine learning algorithms are based on optimization issues. This interaction is also explored in the special topic on machine learning and large-scale optimization. Furthermore, machine learning optimization issues have several unique characteristics that are rarely seen in other optimization contexts. Aside from that, the notions of classical optimization vs machine learning will be discussed. Finally, this study will give an outline of these particular aspects of machine learning optimization.

Keywords

[1] C. C. Aggarwal, Linear algebra and optimization for machine learning, Springer International Publishing,(2020).
[2] G. Y. Ban, N. El Karoui and A. E. Lim, Machine learning and portfolio optimization, Management Science, 3
(2018) 1136-1154.
[3] N. J. Browning, R. Ramakrishnan, O. A. Von Lilienfeld and U. Roethlisberger, U. Genetic optimization of
training sets for improved machine learning models of molecular properties The journal of physical chemistry
letters, 8 (2017) 1351-1359.[4] C. Gambella, B. Ghaddar and J. Naoum-Sawaya, Optimization models for machine learning, a survey. arXiv
preprint arXiv:1901.05331, (2019).
[5] W. T. Garrison and M. D. Petty, An Analysis of Machine Learning Online Training Approaches for Simulation
Optimization, IEEE SoutheastCon, (2019) 1-6.
[6] K. M. Hamdia, X. Zhuang and T. Rabczuk, An efficient optimization approach for designing machine learning
models based on genetic algorithm, Neural Computing and Applications, 33 (2012) 1923-1933.
[7] Mirmozaffari, Mirpouya, et al., A novel machine learning approach combined with optimization models for ecoefficiency evaluation, Applied Sciences, 10.15 (2020) 5210.
[8] C. Song, T. Ristenpart and V. Shmatikov, Machine learning models that remember too much., In Proceedings of
the ACM SIGSAC Conference on computer and communications security, (2017) 587-601.
[9] S. Sun, Z. Cao, H. Zhu and J. Zhao, A survey of optimization methods from a machine learning perspective,
IEEE transactions on cybernetics, 50 (2019) 3668-3681.
[10] Z. Wang and M. O’Boyle, Machine learning in compiler optimization, Proceedings of the IEEE, 11 (2018) 1879-
1901.
[11] J. Wu X. Y. Chen, H. Zhang, L. D. Xiong, H. Lei and S. H. Deng, Hyperparameter optimization for machine
learning models based on Bayesian optimization, Journal of Electronic Science and Technology, 1 (2019) 26-40.
[12] L. Yang and A. Shami, On hyperparameter optimization of machine learning algorithms: Theory and practice,
Neurocomputing, 415 (2020) 295-316.
International Journal of Nonlinear Analysis and Applications
Volume 12, Issue 2
November 2021
Pages 1453-1461
  • Receive Date: 06 March 2021
  • Revise Date: 19 May 2021
  • Accept Date: 12 June 2021