Modified BFGS Update (H-Version) Based on the Determinant Property of Inverse of Hessian Matrix for Unconstrained Optimization

The study presents the modification of the Broyden-Flecher-Goldfarb-Shanno (BFGS) update (HVersion) based on the determinant property of inverse of Hessian matrix (second derivative of the objective function), via updating of the vector s ( the difference between the next solution and the current solution), such that the determinant of the next inverse of Hessian matrix is equal to the determinant of the current inverse of Hessian matrix at every iteration. Moreover, the sequence of inverse of Hessian matrix generated by the method would never approach a near-singular matrix, such that the program would never break before the minimum value of the objective function is obtained. Moreover, the new modification of BFGS update (H-version) preserves the symmetric property and the positive definite property without any condition.


Introduction:
Optimization is the great importance in the various sciences, which have included different fields and aspects. It is an important part of the sciences of mathematics and physics, as well as their importance in engineering, especially mechanical engineering, electricity, management, economy, population growth, weather and other natural phenomena.
Several attempts were made to other quasi-Newton equation to get a better approximation of the inverse of Hessian matrix.
(1) proposed a modified quasi-Newton equation which uses both gradient and function value information in order to yield a higher order accuracy for approximating the second curvature of an objective function. (2) considered a modified Broyden family which includes the BFGS update. (3) modified the BFGS update based on the new quasi-Newton equation, where is a matrix. (4) modified SR1 update based on Zhang-Xu's condition and provided that the update preserves the symmetric and positive definite property and they also provided the global and superlinear convergence of the proposed method. (5) proposed the modified DFP update based on Zhang-Xu's condition and provided the global and superlinear convergence of the proposed method. (6) proposed the modified BFGS method for solving the system of non-linear equations by using Taylor theorem, this proposed method is derivative-free, so the gradient information is not needed at each iteration. (7) proposed a modified quasi-Newton (secant) equation to get a more accurate approximation of the second curvature of the objective function by using Chain rule. Then, is the next solution, = ∇ ( +1 ) − ∇ ( ), ∇ is the gradient of the objective function f , and ∈ . The problem is to solve equation (1) by producing a sequence of symmetric and positive definite (without condition) inverse of Hessian matrix, which never converges to a near-singular matrix that makes the numerical computation break before the minimizer is obtained due to the singularity of the inverse of Hessian matrix numerically. The best solution of this problem is fixing the value of the determinant of inverse of Hessian matrix to be considerably far from zero at every iteration, so that the program does not break prior to obtaining the minimizer.

Modified BFGS Update (H-version):
Consider the BFGS (H-version) update (9), By extended quasi-Newton equation Based on equation (3), the Quasi-Newton equation becomes as follows: The solution of equation. (4) is This is called Modified BFGS update (Hversion), to determine , the following lemmas are necessary:

The Convergence of modified BFGS update (H-version):
In this section, the global convergence for modified BFGS update under exact line search was introduced. The following assumption is needed: : → is twice continuously differentiable on convex set ⊆ . b. ( ) is uniformly convex, i.e. , there exist a positive constants m and M such that for all ∈ ( ) = { : ( ) ≤ ( 0 )}, which is convex, and ‖ ‖ 2 ≤ ∇ 2 ( ) ≤ ‖ ‖ 2 , ∀ ∈ and 0 is the starting point. Case II. If → 0, then ∃ 1 > 0 ∋ ∀ > 1 then, , therefore for a sufficient large k 0 < ψ( +1 ) < ψ( 0 ) + + ∑ Ln( 2 ) − 2 ( 1 =0 − 1 ) < 0 which contradiction, and the proof is complete Numerical Experiments: This section is devoted to the numerical experiments aimed at assessing whether the modified BFGS update (H-version) algorithm provides improvements on the corresponding Standard BFGS update algorithm. The program was written in MATLAB with single precision. The test functions were commonly used unconstrained test problems with the same starting point, a summary of which is given in Table 1.

Conclusion:
In this paper, the BFGS update (Hversion) was modified to preserve the determinant value of the next inverse of Hessian matrix at each iteration equal to the determinant of the current inverse of Hessian matrix and guarantee the strong positive definite property that the Hessian matrix never near singular at each iteration which make the computation continue until the objective function terminate at the minimum.