User talk:Wallers

Welcome!

Hello, Wallers, and welcome to Wikipedia! Thank you for your contributions. I hope you like the place and decide to stay. Here are some pages that you might find helpful:

I hope you enjoy editing here and being a Wikipedian! Please sign your name on talk pages using four tildes ~~~~, which will automatically produce your name and the date.

If you need help, check out Wikipedia:Questions, ask me on my talk page, or place {{helpme}} on your talk page and ask your question there. Again, welcome!

Addbot (talk) 02:10, 20 February 2009 (UTC)[reply]

Considering changes to numerical methods for linear least squares page

I dislike the discussion of the QR decomposition to solve the linear least squares problem. I much prefer Susan Blackford's discussion on the Lapack pages: http://netlib.org/lapack/lug/node40.html. I'm considering changing it to something like this, in line with her discussion.

The QR decomposition allows us to decompose the matrix X into

X=QR

where $Q$ is an $m\times m$ orthogonal matrix and $R$ is an $m\times n$ upper triangular matrix. Because $X$ is overdetermined, $m>n$ , and we have a special case of the QR decomposition where

$X={\begin{bmatrix}Q_{1},Q_{2}\end{bmatrix}}{\begin{bmatrix}R_{n}\\\mathbf {0} \end{bmatrix}}$ .

The goal of the linear least squares solution is to find $\mathbf {\beta }$ which minimizes $||\mathbf {y} -X\mathbf {\beta } ||_{2}$ . Multiplying by an orthogonal matrix does not alter the L2 norm, so

${\begin{aligned}||\mathbf {y} -X\mathbf {\beta } ||_{2}&=||Q^{T}\mathbf {y} -Q^{T}X\mathbf {\beta } ||_{2}\\&=\left|\left|{\begin{bmatrix}Q_{1}^{T}\mathbf {y} -R_{n}\mathbf {\beta } \\Q_{2}^{T}\mathbf {y} \end{bmatrix}}\right|\right|_{2}\end{aligned}}$

The upper portion can be solved for $\mathbf {\beta }$

$\mathbf {\beta } =R_{n}^{-1}Q_{1}^{T}\mathbf {y}$ ,

allowing us to calculate the predicted values

${\hat {\mathbf {y} }}=X\mathbf {\beta }$ .

The residual sum of squares can be either calculated by taking the difference of the predicted and actual values

$||\mathbf {r} ||_{2}^{2}=||\mathbf {y} -{\hat {\mathbf {y} }}||_{2}^{2}$

or by realizing the residual sum of squares is equal to the portion which was independent of $\mathbf {\beta }$

${\begin{aligned}||\mathbf {r} ||_{2}^{2}&=||\mathbf {y} -X\mathbf {\beta } ||_{2}^{2}\\&=\left|\left|{\begin{bmatrix}\mathbf {0} \\Q_{2}^{T}\mathbf {y} \end{bmatrix}}\right|\right|_{2}^{2}\\&=||Q_{2}^{T}\mathbf {y} ||_{2}^{2}.\end{aligned}}$

Wallers (talk) 15:49, 17 June 2010 (UTC)[reply]