Quasi-Newton least squares method

In numerical analysis, the quasi-Newton least squares method is a quasi-Newton method for finding roots in $n$ variables. It was originally described by Rob Haelterman et al. in 2009.

Newton's method for solving $f(x)=0$ uses the Jacobian matrix, $J$, at every iteration. However, computing this Jacobian is a difficult (sometimes even impossible) and expensive operation. The idea behind the quasi-Newton least squares method is to build up an approximate Jacobian based on known input–output pairs of the function $f$.

Haelterman et al. also showed that when the quasi-Newton least squares method is applied to a linear system of size $n\times n$, it converges in at most $n+1$ steps, although like all quasi-Newton methods, it may not converge for nonlinear systems.

The method is closely related to the quasi-Newton inverse least squares method.