Lucas–Kanade Optical Flow Method

From Wikipedia, the free encyclopedia

In computer vision, the Lucas–Kanade method is a two-frame differential method for optical flow estimation developed by Bruce D. Lucas and Takeo Kanade. It introduces an additional term to the optical flow by assuming the flow to be constant in a local neighbourhood around the central pixel under consideration at any given time.

[edit] Concept

The Lucas–Kanade method is still one of the most popular versions of two-frame differential methods for motion estimation (which is also called optical flow estimation). The solution assumes a locally constant flow. The method is based upon the Optical Flow equation. The additional constraint needed for the estimation of the flow field is introduced in this method by assuming that the flow $(V x, V y)$ is constant in a small window of size $m \times m$ with $m > 1$ , which is centered at Pixel $x, y$ and numbering the pixels within as $1... n$ , $n = m 2$ , a set of equations can be found:

$I_{x_1} V_x + I_{y_1} V_y = -I_{t_1}$

$I_{x_2} V_x + I_{y_2} V_y = -I_{t_2}$

$\vdots$

$I_{x_n} V_x + I_{y_n} V_y = -I_{t_n}$

With this there are more than two equations for the two unknowns and thus the system is over-determined. Hence:

$\begin{bmatrix} I_{x_1} & I_{y_1} \\ I_{x_2} & I_{y_2} \\ \vdots & \vdots \\ I_{x_n} & I_{y_n} \end{bmatrix} \begin{bmatrix} V_x\\ V_y \end{bmatrix} = \begin{bmatrix} -I_{t_1}\\ -I_{t_2}\\ \vdots \\ -I_{t_n} \end{bmatrix}$

or

$A\vec{v}=-b$

[edit] Possible Solution

To solve the over-determined system of equations, besides other methods, the least squares method can also be used:

$A^TA\vec{v}=A^T(-b)$ or

$\vec{v}=(A^TA)^{-1}A^T(-b)$

or

$\begin{bmatrix} V_x\\ V_y \end{bmatrix} = \begin{bmatrix} \sum I_{x_i}^2 & \sum I_{x_i}I_{y_i} \\ \sum I_{x_i}I_{y_i} & \sum I_{y_i}^2 \\ \end{bmatrix}^{-1} \begin{bmatrix} -\sum I_{x_i}I_{t_i} \\ -\sum I_{y_i}I_{t_i} \end{bmatrix}$

with the sums running from i=1 to n.

This means that the optical flow can be found by calculating the derivatives of the image in along all dimensions. A weighting function W(i,j), with $i,j \in [1,m]$ should be added to give more prominence to the central pixel of the window. Gaussian functions are preferred for this purpose however, other functions or weighting schemes are also possible. Besides for computing local translations, the flow model can also be extended to affine image deformations.

[edit] Improvements

As generally more equations are available for flow estimation then needed (over determined system) the Lucas-Kanade algorithm can be used in combination with statistical methods to improve the performance in presence of outliers as in noisy images. A statistical analysis marks the outliers and the flow is then estimated based on the remaining equations or weighted accordingly.

When applied to image registration, such as stereo matching or images with large displacements, the Lucas–Kanade method is usually carried out in a coarse-to-fine iterative manner, in such a way that the spatial derivatives are first computed at a coarse scale in scale-space (or a pyramid), one of the images is warped by the computed deformation, and iterative updates are then computed at successively finer scales.

[edit] Properties

One of the characteristics of the Lucas–Kanade algorithm, and that of other local optical flow algorithms, is that it does not yield a very high density of flow vectors, i.e. the flow information fades out quickly across motion boundaries and the inner parts of large homogenous areas show little or no motion. Its advantage is the comparative robustness in presence of noise.

[edit] See also

[edit] References

Lucas B D and Kanade T 1981, An iterative image registration technique with an application to stereo vision. Proceedings of Imaging understanding workshop, pp 121--130 (pdf).
Lucas B D 1984, Generalized Image Matching by the Method of Differences, doctoral dissertation
KLT: An Implementation of the Kanade–Lucas–Tomasi Feature Tracker
[1]: Takeo Kanade

[edit] External links

The image stabilizer plugin for ImageJ based on the Lucas–Kanade method
Mathworks Lucas-Kanade Matlab implementation of inverse and normal affine Lucas-Kanade

Lucas–Kanade Optical Flow Method

From Wikipedia, the free encyclopedia

Contents

[edit] Concept

[edit] Possible Solution

[edit] Improvements

[edit] Properties

[edit] See also

[edit] References

[edit] External links

Views

Personal tools

Navigation

Search

Interaction

Toolbox

Languages