Modifications for Quasi-Newton Method and Its Spectral Algorithm for Solving Unconstrained Optimization Problems
DOI:
https://doi.org/10.21123/bsj.2023.8020Keywords:
BFGS algorithm, Inexact line search, Numerical optimization, Spectral quasi-Newton method, Unconstrained optimizationAbstract
In this paper, two modifications for spectral quasi-Newton algorithm of type BFGS are imposed. In the first algorithm, named SQNEI, a certain spectral parameter is used in such a step for BFGS algorithm differs from other presented algorithms. The second algorithm, SQNEv-Iv, has both new parameter position and value suggestion. In SQNEI and SQNEv-Iv methods, the parameters are involved in a search direction after an approximated Hessian matrix is updated. It is provided that two methods are effective under some assumptions. Moreover, the sufficient descent property is proved as well as the global and superlinear convergence for SQNEv-Iv and SQNEI. Both of them are superior the standard BFGS (QNBFGS) and previous spectral quasi-Newton (SQNLC). However, SQNEv-Iv is outstanding SQNEI if it is convergent to the solution. This means that, two modified methods are in the race for the more efficiency method in terms less iteration numbers and consuming time in running CPU. Finally, numerical results are presented for the four algorithms by running list of test problems with inexact line search satisfying Armijo condition.
Received 29/10/2022
Revised 29/05/2023
Accepted 31/05/2023
Published Online First 20/11/2023.
References
Andrei N. Nonlinear conjugate gradient methods for unconstrained optimization. Berlin, Heidelberg: Springer; 2020 Jun 23.
Mahmood SS. Modified BFGS Update (H-Version) Based on the Determinant Property of Inverse of Hessian Matrix for Unconstrained Optimization. Baghdad Sci. J. 2020;17(3(Suppl.)): 0994. https://doi.org/10.21123/bsj.2020.17.3(Suppl.).0994
Mahmood SS, Eidi JH, Jasem JA. Positive Definiteness of Symmetric Rank 1 (H-Version) Update for Unconstrained Optimization. Baghdad Sci. J. 2022;19(2):297-303. https://doi.org/10.21123/bsj.2022.19.2.0297.
Cheng W, Li DH. Spectral Scaling BFGS Method. J Optim Theory Appl. 2010;146:305-319. https://doi.org/10.1007/s10957-010-9652-y
Nakayama S, Narushima Y, Yabe H. A memoryless symmetric rank-one method with sufficient descent property for unconstrained optimization. J Oper Res Soc Japan. 2018;61(1):53-70. https://doi.org/10.15807/jorsj.61.53
Nakayama S, Narushima Y. GLOBAL CONVERGENCE OF A PROXIMAL MEMORYLESS SYMMETRIC RANK ONE METHOD FOR MINIMIZING COMPOSITE FUNCTIONS. Proc of Int Conf Nonlinear Analysis and Convex Analysis & Int Conf on Optimization: Techniques and Applications-II. Hakodate, Japan. 2019:99-108.
Nakayama S. A hybrid method of three-term conjugate gradient method and memoryless quasi-newton method for unconstrained optimization. SUT J Math. 2018;54(1):79-98. https://doi.org/10.55937/sut/1547386572
Lv J, Deng S, Wan Z. An Efficient Single-Parameter Scaling Memoryless Broyden-Fletcher-Goldfarb-Shanno Algorithm for Solving Large Scale Unconstrained Optimization Problems. IEEE Access. 2020;8:85664-85674. https://doi.org/10.1109/ACCESS.2020.2992340
Al-Naemi GM, Sheekoo AH. New scaled algorithm for non-linear conjugate gradients in unconstrained optimization. Indones J Electr Eng Comput Sci. 2021;24(3):1589-1595. https://doi.org/10.11591/ijeecs.v24.i3.pp1589-1595
Wang L, Cao M, Xing F, Yang Y. The new spectral conjugate gradient method for large-scale unconstrained optimisation. J Inequalities Appl. 2020;2020(1). https://doi.org/10.1186/s13660-020-02375-z
Al-Arbo AA, Al-Kawaz RZ. A fast spectral conjugate gradient method for solving nonlinear optimization problems. Indones J Electr Eng Comput Sci. 2021;21(1):429-439. https://doi.org/10.11591/ijeecs.v21.i1.pp429-439
Qasim AM, Salih ZF. New Spectral Idea for Conjugate Gradient Methods and Its Global Convergence Theorems. 2022;15(2):784-795.
Nakayama S, Narushima Y, Nishio H, Yabe H. An active-set memoryless quasi-Newton method based on a spectral-scaling Broyden family for bound constrained optimization. Results Control Optim. 2021;3:100012. https://doi.org/10.1016/j.rico.2021.100012
Ibrahim SM, Yakubu UA, Mamat M. Application of spectral conjugate gradient methods for solving unconstrained optimization problems. Int J Optim Control Theor Appl. 2020;10(2):198-205. https://doi.org/10.11121/IJOCTA.01.2020.00859
Ibrahim MAH, Mamat M, Leong WJ. The Hybrid BFGS-CG Method in Solving Unconstrained Optimization Problems. Abstr Appl Anal. 2014;2014:1-6. https://doi.org/10.1155/2014/507102
Han L, Neumann M. Combining Quasi-Newton and Steepest Descent Directions. January 2003; 1-30.
Andrei N. An Unconstrained Optimization Test Functions Collection. Adv Model Optim. 2008;10(1):147-161.
Jamil M, Yang XS. A literature survey of benchmark functions for global optimisation problems. Int J Math Model Numer Optim. 2013;4(2):150-194. https://doi.org/10.1504/IJMMNO.2013.055204
Downloads
Issue
Section
License
Copyright (c) 2023 Baghdad Science Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
