Structure from motion

In computer vision structure from motion refers to the process of finding the three-dimensional structure of an object by analyzing local motion signals over time. In Vision Science (visual perception), Structure from motion (SfM), refers to the general phenomenon by which humans (and other animals) can recover 3-D structure from the projected 2D (retinal) motion field of a moving object.

Obtaining 3D information from 2D images

Digital Surface Model of motorway interchange construction site

Bezmiechowa airfield 3D Digital Surface Model extracted from data collected during 30min flight of Pteryx UAV

Humans perceive a lot of information about the three-dimensional structure in their environment by moving through it. When the observer moves and the objects around him move, information is obtained from images sensed over time.

Finding structure from motion presents a similar problem as finding structure from stereo vision. In both instances, the correspondence between images and the reconstruction of 3D object needs to be found.

To find correspondence between images, features such as corner points (edges with gradients in multiple directions) need to be tracked from one image to the next. The feature trajectories over time are then used to reconstruct their 3D positions and the camera's motion.

Structure from motion is related to the kinetic depth effect in perception whereby subjects viewing the shadow cast by a wire frame or other structure in rotation, perceive the full three-dimensional structure of the object, whereas when viewing the shadow of a static object they perceive only its two-dimensional projection. Actually, the terms "kinetic depth effect" and "structure from motion" are usually used indistinctly in the vision science literature.

See also

• Match moving
• Motion field
• Epipolar geometry
• Tomasi–Kanade factorization

Further reading

• Richard Hartley and Andrew Zisserman (2003). Multiple View Geometry in Computer Vision. Cambridge University Press. ISBN 0-521-54051-8. 

• Olivier Faugeras and Quang-Tuan Luong and Theodore Papadopoulo (2001). The Geometry of Multiple Images. MIT Press. ISBN 0-262-06220-8. 
• Yi Ma, S. Shankar Sastry, Jana Kosecka, Stefano Soatto, Jana Kosecka (November 2003). An Invitation to 3-D Vision: From Images to Geometric Models. Interdisciplinary Applied Mathematics Series, #26. Springer-Verlag New York, LLC. ISBN 0-387-00893-4. 

External links

Structure from Motion software toolboxes

1. FIT3D (From Images to 3D) Toolbox by Isaac Esteban
2. Structure from Motion toolbox for Matlab by Vincent Rabaud
3. Matlab Functions for Multiple View Geometry by Andrew Zissermann
4. Structure and Motion Toolkit by Phil Torr
5. Non Rigid Structure from Motion in trajectory space by Ijaz Akhter.
6. Matlab Code for Non-Rigid Structure from Motion using Factorisation by Lorenzo Torresani Kooni
7. Bundler - Structure from Motion for Unordered Photo Collections by Noah Snavely
8. Voodoo Camera Tracker: A tool for the integration of virtual and real scenes, by Laboratorium für Informationstechnologie, University of Hannover
9. Libmv - A C++ Structure from Motion library
10. VisualSFM - A Visual Structure from Motion System