Jump to content

Quasi-Newton method

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by AbnerCYH (talk | contribs) at 03:02, 12 November 2006. The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

In Optimization (mathematics), Quasi-Newton method is a well-known algorithm for finding local maxima and local minima of functions like Newton's method but it approximate the Hessian matrix for accelerating the iteration.

The first Quasi-Newton Algorithm was proposed by W.C. Davidon, a physicist working at Argonne National Laboratory. Like the story of QuickSort, Davidon got the trouble of applying optimization algorithms on his research, thus he decided to improve such algorithm. Eventually he developed the first quasi-Newton algorithm in the mid 1950s.

However, such seminal idea is not accpeted for publications, it remained as a technical report until 1991.

The most popular quasi-Newton algorithms are DFP algorithm and BFGS method. The former was developed by Davidon in 1959 and was subsequently modified by Fletcher and Powell in 1963; the later was was suggested independently by Broyden, Fletcher, Goldfarb, and Shanno, in 1970.

References

  • Eventually W.C. Davidon's paper published. William C. Davidon, Variable Metric Method for Minimization, SIOPT Volume 1 Issue 1, Pages 1-17, 1991.
  • Nocedal, Jorge & Wright, Stephen J. (1999). Numerical Optimization. Springer-Verlag. ISBN 0-387-98793-2.
  • Edwin K.P.Chong and Stanislaw H.Zak, An Introduction to Optimization 2ed, John Wiley & Sons Pte. Ltd. August 2001.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Quasi-Newton_method&oldid=87262481"