MODIFIED LIMITED MEMORY BFGS METHOD WITH NONMONOTONE LINE SEARCH FOR UNCONSTRAINED OPTIMIZATION

- Journal title : Journal of the Korean Mathematical Society
- Volume 47, Issue 4, 2010, pp.767-788
- Publisher : The Korean Mathematical Society
- DOI : 10.4134/JKMS.2010.47.4.767

Title & Authors

MODIFIED LIMITED MEMORY BFGS METHOD WITH NONMONOTONE LINE SEARCH FOR UNCONSTRAINED OPTIMIZATION

Yuan, Gonglin; Wei, Zengxin; Wu, Yanlin;

Yuan, Gonglin; Wei, Zengxin; Wu, Yanlin;

Abstract

In this paper, we propose two limited memory BFGS algorithms with a nonmonotone line search technique for unconstrained optimization problems. The global convergence of the given methods will be established under suitable conditions. Numerical results show that the presented algorithms are more competitive than the normal BFGS method.

Keywords

limited memory BFGS method;optimization;nonmonotone;global convergence;

Language

English

Cited by

1.

2.

3.

4.

5.

6.

8.

9.

References

1.

C. G. Broyden, J. E. Dennis Jr, and J. J. More, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl. 12 (1973), 223-245.

2.

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal. 26 (1989), no. 3, 727-739.

3.

R. H. Byrd, J. Nocedal, and R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods, Math. Programming 63 (1994), no. 2, Ser. A, 129-156.

4.

R. Byrd, J. Nocedal, and Y. Yuan, Global convergence of a class of quasi-Newton methods on convex problems, SIAM J. Numer. Anal. 24 (1987), no. 5, 1171-1190.

6.

Y. Dai and Q. Ni, Testing different conjugate gradient methods for large-scale unconstrained optimization, J. Comput. Math. 21 (2003), no. 3, 311-320.

7.

W. C. Davidon, Variable metric methods for minimization, A. E. C. Research and Development Report ANL-599, 1959.

8.

R. Dembo and T. Steihaug, Truncated Newton algorithms for large-scale unconstrained optimization, Math. Programming 26 (1983), no. 2, 190-212.

9.

J. E. Dennis Jr. and J. J. More, Quasi-Newton methods, motivation and theory, SIAM Rev. 19 (1977), no. 1, 46-89.

10.

J. E. Dennis Jr. and J. J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974), 549-560.

11.

J. E. Dennis Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall Series in Computational Mathematics. Prentice Hall, Inc., Englewood Cliffs, NJ, 1983.

12.

L. C. W. Dixon, Variable metric algorithms: necessary and sufficient conditions for identical behavior of nonquadratic functions, J. Optimization Theory Appl. 10 (1972), 34-40.

13.

E. D. Dolan and J. J. More, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), no. 2, Ser. A, 201-213.

14.

R. Fletcher, Practical Methods of Optimization, Second edition. A Wiley-Interscience Publication. John Wiley & Sons, Ltd., Chichester, 1987.

15.

N. I. M. Gould, D. Orban, and Ph. L. Toint, CUTEr (and SifDec), a constrained and unconstrained testing environment, revisite, ACM Transactions on Mathematical Software 29 (2003), 373-394.

16.

A. Griewank, The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients, Math. Programming 50 (1991), no. 2, (Ser. A), 141-175.

17.

A. Griewank and Ph. L. Toint, Local convergence analysis for partitioned quasi-Newton updates, Numer. Math. 39 (1982), no. 3, 429-448.

18.

L. Grippo, F. Lamparillo, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), no. 4, 707-716.

19.

L. Grippo, F. Lamparillo, and S. Lucidi, A truncated Newton method with nonmonotone line search for unconstrained optimization, J. Optim. Theory Appl. 60 (1989), no. 3, 401-419.

20.

L. Grippo, F. Lamparillo, and S. Lucidi, A class of nonmonotone stabilization methods in unconstrained optimization, Numer. Math. 59 (1991), no. 8, 779-805.

21.

J. Y. Han and G. H. Liu, Global convergence analysis of a new nonmonotone BFGS algorithm on convex objective functions, Comput. Optim. Appl. 7 (1997), no. 3, 277-289.

22.

D. Li and M. Fukushima, A modified BFGS method and its global convergence in non-convex minimization, J. Comput. Appl. Math. 129 (2001), no. 1-2, 15-35.

23.

L. Grippo, F. Lamparillo, and S. Lucidi, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim. 11 (2001), no. 4, 1054-1064.

24.

G. Li, C. Tang, and Z.Wei, New conjugacy condition and related new conjugate gradient methods for unconstrained optimization, J. Comput. Appl. Math. 202 (2007), no. 2, 523-539.

25.

G. H. Liu and J. Y. Han, Notes on the general form of stepsize selection, OR and Decision Making I (1992), 619-624.

26.

G. H. Liu and J. Y. Han, Global convergence Analysis of the variable metric algorithm with a generalized Wolf linesearch, Technical Report, Institute of Applied Mathematics, Academia Sinica, Beijing, China, no. 029, 1993.

27.

G. H. Liu, J. Y. Han, and D. F. Sun, Global convergence of the BFGS algorithm with nonmonotone linesearch, Optimization 34 (1995), no. 2, 147-159.

28.

G. H. Liu and J. M. Peng, The convergence properties of a nonmonotonic algorithm, J. Comput. Math. 1 (1992), 65-71.

29.

W. F. Mascarenhas, The BFGS method with exact line searches fails for non-convex objective functions, Math. Program. 99 (2004), no. 1, Ser. A, 49-61.

30.

J. J. More, B. S. Garbow, and K. E. Hillstrome, Testing unconstrained optimization software, ACM Trans. Math. Software 7 (1981), no. 1, 17-41.

31.

S. G. Nash, A survey of truncated-Newton methods, Numerical analysis 2000, Vol. IV, Optimization and nonlinear equations. J. Comput. Appl. Math. 124 (2000), no. 1-2, 45-59.

32.

M. J. D. Powell, On the convergence of the variable metric algorithm, J. Inst. Math. Appl. 7 (1971), 21-36.

33.

M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches, Nonlinear programming (Proc. Sympos., New York, 1975), pp. 53–72. SIAM-AMS Proc., Vol. IX, Amer. Math. Soc., Providence, R. I., 1976.

34.

M. J. D. Powell, A new algorithm for unconstrained optimization, 1970 Nonlinear Programming (Proc. Sympos., Univ. of Wisconsin, Madison, Wis., 1970) pp. 31-65 Academic Press, New York.

35.

M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim. 7 (1997), no. 1, 26-33.

36.

J. Schropp, A note on minimization problems and multistep methods, Numer. Math. 78 (1997), no. 1, 87-101.

37.

J. Schropp, One-step and multistep procedures for constrained minimization problems, IMA J. Numer. Anal. 20 (2000), no. 1, 135-152.

38.

Ph. L. Toint, Global convergence of the partitioned BFGS algorithm for convex partially separable optimization, Math. Programming 36 (1986), no. 3, 290-306.

39.

D. J. van Wyk, Differential optimization techniques, Appl. Math. Modelling 8 (1984), no. 6, 419-424.

40.

M. N. Vrahatis, G. S. Androulakis, J. N. Lambrinos, and G. D. Magolas, A class of gradient unconstrained minimization algorithms with adaptive stepsize, J. Comput. Appl. Math. 114 (2000), no. 2, 367-386.

41.

Z. Wei, G. Li, and L. Qi, New quasi-Newton methods for unconstrained optimization problems, Appl. Math. Comput. 175 (2006), no. 2, 1156-1188.

42.

Z. Wei, G. Yu, G. Yuan, and Z. Lian, The superlinear convergence of a modified BFGS-type method for unconstrained optimization, Comput. Optim. Appl. 29 (2004), no. 3, 315-332.

43.

G. L. Yuan, Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optim. Lett. 3 (2009), no. 1, 11-21.

44.

G. L. Yuan and X. W. Lu, A new line search method with trust region for unconstrained optimization, Comm. Appl. Nonlinear Anal. 15 (2008), no. 1, 35-49.

45.

G. L. Yuan and X. W. Lu, A modified PRP conjugate gradient method, Ann. Oper. Res. 166 (2009), 73-90.

46.

G. L. Yuan, X. Lu, and Z. Wei, A conjugate gradient method with descent direction for unconstrained optimization, J. Comput. Appl. Math. 233 (2009), no. 2, 519-530.

47.

Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, 1999.

48.

G. L. Yuan and Z. X. Wei, New line search methods for unconstrained optimization, J. Korean Statist. Soc. 38 (2009), no. 1, 29-39.

49.

G. L. Yuan and Z. X. Wei, The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions, Acta Math. Sin. (Engl. Ser.) 24 (2008), no. 1, 35-42.