# Reprojection error

The reprojection error is a geometric error corresponding to the image distance between a projected point and a measured one. It is used to quantify how closely an estimate of a 3D point $\hat{\mathbf{X}}$ recreates the point's true projection $\mathbf{x}$. More precisely, let $\mathbf{P}$ be the projection matrix of a camera and $\hat{\mathbf{x}}$ be the image projection of $\hat{\mathbf{X}}$, i.e. $\hat{\mathbf{x}}=\mathbf{P} \, \hat{\mathbf{X}}$. The reprojection error of $\hat{\mathbf{X}}$ is given by $d(\mathbf{x}, \, \hat{\mathbf{x}})$, where $d(\mathbf{x}, \, \hat{\mathbf{x}})$ denotes the Euclidean distance between the image points represented by vectors $\mathbf{x}$ and $\hat{\mathbf{x}}$.
Minimizing the reprojection error can be used for estimating the error from point correspondences between two images. Suppose we are given 2D to 2D point imperfect correspondences $\{\mathbf{x_i} \leftrightarrow \mathbf{x_i}'\}$. We wish to find a homography $\hat{\mathbf{H}}$ and pairs of perfectly matched points $\hat{\mathbf{x_i}}$ and $\hat{\mathbf{x}}_i'$, i.e. points that satisfy $\hat{\mathbf{x_i}}' = \hat{H}\mathbf{\hat{x}_i}$ that minimize the reprojection error function given by
$\sum_i d(\mathbf{x_i}, \hat{\mathbf{x_i}})^2 + d(\mathbf{x_i}', \hat{\mathbf{x_i}}')^2$
So the correspondences can be interpreted as imperfect images of a world point and the reprojection error quantifies their deviation from the true image projections $\hat{\mathbf{x_i}}, \hat{\mathbf{x_i}}'$