International Journal of Nonlinear Analysis and Applications
New adapted spectral method for solving stochastic optimal control problem
Document Type : Research Paper
Authors
Mathematical Analysis and Applications Laboratory, Departement of Mathematics, Faculty of Mathematics and Informatics, Mohamed El Bachir El Ibrahimi university of Bordj Bou Arreridj, El Anasser, 34030, Algeria
Abstract
Optimal control theory is a branch of mathematics. It is developed to find optimal ways to control a dynamic system. In 1957, R.Bellman applied dynamic programming to solve optimal control of discrete-time systems. His procedure resulted in closed-loop, generally nonlinear, and feedback schemes. Optimal control problems which will be tackled involve the minimization of a cost function subject to constraints on the state vector and the control. Lagrange multipliers provide a method of converting a constrained minimization problem into an unconstrained minimization problem of higher order. The necessary condition for optimality can be obtained as the solution of the unconstrained optimization problem of the Lagrange function and the bordered Hessian matrix is used for the second-derivative test. A spectral method for solving optimal control problems is presented. The method is based on Bernoulli polynomials approximation. By using the Bernoulli operational matrix of integration and the Lagrangian function, stochastic optimal control is transformed into an optimisation problem, where the unknown Bernoulli coefficients are determined by using Newton's iterative method. The convergence analysis of the proposed method is given. The simulation results based on the Monte-Carlo technique prove the performance of the proposed method. Some error estimations are provided and illustrative examples are also included to demonstrate the efficiency and applicability of the proposed method.
Keywords
[1] M. Annunziato and A. Borzi, Optimal control of probability density functions of stochastic processes, Math. Model. Anal. 15 (2010), no. 4, 393–407.
[2] M. Annunziato and A. Borzi, A Fokker-Planck control framework for multidimensional stochastic processes, J. Comput. Appl. Math. 237 (2013), no. 1, 487–507.
[3] S. Bazm, Bernoulli polynomials for the numerical solution of some classes of linear and nonlinear integral equations, J. Comput. Appl. Math. 275 (2015), 44–60.
[4] R. Bellman, Dynamic programming and stochastic control processes, Inf. Control 1 (1958), no. 3, 228–239.
[5] A.H. Bhrawy, E. Tohidi, and F. Soleymani, A new Bernoulli matrix method for solving high-order linear and nonlinear Fredholm integro-differential equations with piecewise intervals, Appl. Math. Comput. 219 (2012), no. 2, 482–497.
[6] J.M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim. 14 (1976), no. 3, 419–444.
[7] J.M. Bismut, Controle des Systemes Lineaires Quadratiques: Applications de l’integrale Stochastique, Springer-Verlag, Berlin, 1978.
[8] Z. Chen, X. Feng, S. Liuand, and W. Zhang, Bang-bang control for a class of optimal stochastic control problems with symmetric cost functional, Automatica 149 (2023), 110849.
[9] S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chinese Ann. Math. 21 (2000), no. 3, 323–338.
[10] Y. Chen and Y. Zhu, Optimistic value model of indefinite LQ optimal control for discrete-time uncertain systems, Asian J. Control 20 (2018), no. 1, 495–510.
[11] F.A. Costabile and F. Dell’Accio, Expansion over a rectangle of real functions in Bernoulli polynomials and applications, BIT Numer. Math. 43 (2001), 451–464.
[12] L. Deng and Y. Zhu, An uncertain optimal control model with n jumps and application, Comput. Sci. Inf. Syst. 9 (2012), no. 4, 1453–1468.
[13] J.C. Doyle and B.A. Francis and A.R. Tannenbaum, Feedback Control Theory, Courier Corporation, 2013.
[14] N. Du, J. Shi, and W. Liu, An effective gradient projection method for stochastic optimal control, Int. J. Numer. Anal. Model. 10 (2013), no. 4, 757–774.
[15] R. Elliott, X. Li, and Y.H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica 49 (2013), no. 11, 3222–3233.
[16] J. Engwerda, LQ Dynamic Optimization and Differential Games, John Wiley & Sons, 2005.
[17] N. Ghaderi and M.H. Farahi, The numerical solution of nonlinear optimal control problems by using operational matrix of Bernstein polynomials, Math. Anal. Convex Optim. 2 (2021), no. 1, 11–27.
[18] R. Herzallah, Generalised probabilistic control design for uncertain stochastic control systems, Asian J. Control 20 (2018), no. 6, 2065–2074.
[19] S. Ji, S. Peng, Y. Peng, and X. Zhang, Solving stochastic optimal control problem via stochastic maximum principle with deep learning method, J. Sci. Comput. 93 (2022), no. 1, 30.
[20] B. Kafash and A. Delavarkhalafi, Restarted state parameterization method for optimal control problems, J. Math. Comput. Sci. 14 (2015), 151–161.
[21] M. Kohlmann and S. Tang, New developments in backward stochastic Riccati equations and their applications, Math. Finance: Workshop Math. Finance Res. Project, Konstanz, Germany, October 5–7, 2000. Birkhauser Basel, 2001, pp. 194–214.
[22] M. Kohlmann and S. Tang, Multidimensional backward stochastic Riccati equations and applications, SIAMJ. Control Optim. 41 (2003), no. 6, 1696–1721.
[23] D.H. Lehmer, A new approach to Bernoulli polynomials, Amer. Math. Month. 95 (1988), no. 10, 905–911.
[24] Q. Lu and T. Wang, Optimal feedback controls of stochastic linear quadratic control problems in infinite dimensions with random coefficients, J. Math. Pures Appl. 173 (2023), 195–242.
[25] Q. Lu, T. Wang, and X. Zhang, Characterization of optimal feedback for stochastic linear quadratic control problems, Prob. Uncert. Quant. Risk 2 (2017), 1–20.
[26] K. Maleknejad, M. Khodabin, and F. Hosseini Shekarabi, Modified block pulse functions for numerical solution of stochastic Volterra integral equations, J. Appl. Math. 2014 (2014).
[27] K. Maleknejad, M. Khodabin, and M. Rostami, Numerical solution of stochastic Volterra integral equations by a stochastic operational matrix based on block pulse functions, Math. Comput. Modell. 55 (2012), no. 3-4, 791–800.
[28] F. Mirzaee and E. Hadadiyan, Numerical solution of Volterra–Fredholm integral equations via modification of hat functions, Appl. Math. Comput. 280 (2016), 110–123.
[29] F. Mirzaee, N. Samadyar, and S.F. Hosseini, A new scheme for solving nonlinear Stratonovich Volterra integral equations via Bernoullis approximation, Appl. Anal. 96 (2017), no. 13, 2163–2179.
[30] E.H. Ouda, The efficient generalized Laguerre parameterization for quadratic optimal control problem, J. College Educ. Al-Mustansyriah Univer. 3 (2014), no. 1812-0380, 263–276.
[31] S. Peng and Z. Wu, Fully coupled forward-backwards stochastic differential equations and applications to optimal control, SIAM J. Control Optim. 37 (1999), no. 3, 825–843.
[32] M. Saffarzadeh, A. Delavarkhalafi, and Z. Nikoueinezhad, Numerical method for solving optimal control problem of stochastic Volterra integral equations using block pulse functions, J. Math. Comput. Sci. 11 (2014), 22–36.
[33] R. Schlosser, A stochastic dynamic pricing and advertising model under risk aversion, J. Rev. Pric. Manag. 14 (2015), 451–468.
[34] Y. Shang, Optimal control strategies for virus spreading in inhomogeneous epidemic dynamics, Canad. Math. Bull. 56 (2013), no. 3, 621–629.
[35] S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAMJ. Control Optim. 53 (2015), no. 2, 1082–1106.
[36] K.L. Teo, D.W. Reid, and I.E. Boyd, Stochastic optimal control theory and its computational methods, Int. J. Syst. Sci. 11 (1980), no. 1, 77–95.
[37] E. Tohidi, A.H. Bhrawy, and K. Erfani, A collocation method based on Bernoulli operational matrix for numerical solution of generalized pantograph equation, Appl. Math. Model. 37 (2013), no. 6, 4283—4294.
[38] N. Touzi and A. Tourin, Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE, Springer Science and Business Media, 2012.
[39] G. Wang, Z. Wu, and J. Xiong, An Introduction to Optimal Control of FBSDE with Incomplete Information, Springer, 2018.
[40] W.M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control Optim. 6 (1968), no. 4, 681–697.
[41] J.L. Wu and T.T. Lee, Robust H∞ control problem for general nonlinear systems with uncertainty, Asian J. Control 5 (2003), no. 2, 168–175.
[42] S. Xing, Y. Liu, and D.Y. Liu, An improved iterative algorithm for solving optimal tracking control problems of stochastic systems, Math. Comput. Simul. 213 (2023), 515–526.
[43] R. Zeghdane, Numerical solution of stochastic integral equations by using Bernoulli operational matrix, Math. Comput. Simul. 165 (2019), 238–254.
[44] H. Zhang and X. Zhang, Stochastic linear quadratic optimal control problems with expectation-type linear equality constraints on the terminal states, Syst. Control Lett. 177 (2023), 105551.
[45] J. Zheng and L. Qiu, On the existence of a mean-square stabilizing solution to a modified algebraic Riccati equation, IFAC Proc. 47 (2014), no. 3, 6988–6993.
[46] K. Zhou and C.J. Doyle, Essentials of Robust Control, Prentice Hall Upper Saddle River, NJ., 1998.
)
Volume 15, Issue 11
November 2024Pages 1-28
- Receive Date: 16 May 2023
- Revise Date: 28 September 2023
- Accept Date: 13 October 2023