Horn–Schunck method

The Horn–Schunck method of estimating optical flow is a global method which introduces a global constraint of smoothness to solve the aperture problem (see Optical Flow for further description).

Mathematical details

The Horn-Schunck algorithm assumes smoothness in the flow over the whole image. Thus, it tries to minimize distortions in flow and prefers solutions which show more smoothness.

The flow is formulated as a global energy functional which is then sought to be minimized. This function is given for two-dimensional image streams as:

${\displaystyle E=\iint \left[(I_{x}u+I_{y}v+I_{t})^{2}+\alpha ^{2}(\lVert \nabla u\rVert ^{2}+\lVert \nabla v\rVert ^{2})\right]{{\rm {d}}x{\rm {d}}y}}$

where ${\displaystyle I_{x}}$, ${\displaystyle I_{y}}$ and ${\displaystyle I_{t}}$ are the derivatives of the image intensity values along the x, y and time dimensions respectively, ${\displaystyle {\vec {V}}=[u(x,y),v(x,y)]^{\top }}$ is the optical flow vector (which is to be solved for), and the parameter ${\displaystyle \alpha }$ is a regularization constant. Larger values of ${\displaystyle \alpha }$ lead to a smoother flow. This functional can be minimized by solving the associated multi-dimensional Euler–Lagrange equations. These are

${\displaystyle {\frac {\partial L}{\partial u}}-{\frac {\partial }{\partial x}}{\frac {\partial L}{\partial u_{x}}}-{\frac {\partial }{\partial y}}{\frac {\partial L}{\partial u_{y}}}=0}$

${\displaystyle {\frac {\partial L}{\partial v}}-{\frac {\partial }{\partial x}}{\frac {\partial L}{\partial v_{x}}}-{\frac {\partial }{\partial y}}{\frac {\partial L}{\partial v_{y}}}=0}$

where ${\displaystyle L}$ is the integrand of the energy expression, giving

${\displaystyle I_{x}(I_{x}u+I_{y}v+I_{t})-\alpha ^{2}\Delta u=0}$

${\displaystyle I_{y}(I_{x}u+I_{y}v+I_{t})-\alpha ^{2}\Delta v=0}$

where subscripts again denote partial differentiation and ${\displaystyle \Delta ={\frac {\partial ^{2}}{\partial x^{2}}}+{\frac {\partial ^{2}}{\partial y^{2}}}}$ denotes the Laplace operator. In practice the Laplacian is approximated numerically using finite differences, and may be written ${\displaystyle \Delta u(x,y)=({\overline {u}}(x,y)-u(x,y))}$ where ${\displaystyle {\overline {u}}(x,y)}$ is a weighted average of ${\displaystyle u}$ calculated in a neighborhood around the pixel at location (x,y). Using this notation the above equation system may be written

${\displaystyle (I_{x}^{2}+\alpha ^{2})u+I_{x}I_{y}v=\alpha ^{2}{\overline {u}}-I_{x}I_{t}}$

${\displaystyle I_{x}I_{y}u+(I_{y}^{2}+\alpha ^{2})v=\alpha ^{2}{\overline {v}}-I_{y}I_{t}}$

which is linear in ${\displaystyle u}$ and ${\displaystyle v}$ and may be solved for each pixel in the image. However, since the solution depends on the neighboring values of the flow field, it must be repeated once the neighbors have been updated. The following iterative scheme is derived using Cramer's rule:

${\displaystyle u^{k+1}={\overline {u}}^{k}-{\frac {I_{x}(I_{x}{\overline {u}}^{k}+I_{y}{\overline {v}}^{k}+I_{t})}{\alpha ^{2}+I_{x}^{2}+I_{y}^{2}}}}$

${\displaystyle v^{k+1}={\overline {v}}^{k}-{\frac {I_{y}(I_{x}{\overline {u}}^{k}+I_{y}{\overline {v}}^{k}+I_{t})}{\alpha ^{2}+I_{x}^{2}+I_{y}^{2}}}}$

where the superscript k+1 denotes the next iteration, which is to be calculated and k is the last calculated result. This is in essence a Matrix splitting method, similar to the Jacobi method, applied to the large, sparse system arising when solving for all pixels simultaneously^{[citation needed]}.

Properties

Advantages of the Horn–Schunck algorithm include that it yields a high density of flow vectors, i.e. the flow information missing in inner parts of homogeneous objects is filled in from the motion boundaries. On the negative side, it is more sensitive to noise than local methods.

See also

• Lucas–Kanade method

References

• B.K.P. Horn and B.G. Schunck, "Determining optical flow." Artificial Intelligence, vol 17, pp 185–203, 1981. Manuscript available on MIT server.

External links

• OpenCV implementation