FreeTrack

Stable release	v2.2 / October 7, 2008
Operating system	Microsoft Windows
Type	Optical motion tracking
License	GNU General Public License
Website	www.free-track.net

FreeTrack is a general purpose optical motion tracking application for Microsoft Windows, released under the GNU General Public License, that can be used with common inexpensive cameras. Its primary focus is head tracking with uses in virtual reality, simulation, video games, 3D modeling, computer aided design and general hands-free computing to improve computer accessibility. Tracking can be made sensitive enough that only small head movements are required so that the user's eyes never leave the screen.

A camera is positioned to observe a rigid point model worn by the user, the points of which need to be isolated from background light by means of physical and software filtering. Motion is tracked with up to six degrees of freedom (6DOF), specifically; yaw, pitch, roll, left/right, up/down and forward/back. Windows compatible video devices like webcams are supported, as well as special support for the Nintendo Wii Remote camera and NaturalPoint cameras (TrackIR, SmartNav and OptiTrack).

FreeTrack can output head tracking data to programs directly using its own open interface, as well as TrackIR, SimConnect and FSUIPC interfaces. Programs that support these interfaces are regarded as being FreeTrack-compatible. FreeTrack can also emulate mouse, keyboard, and joystick (via PPJoy) if a program does not support a direct interface.

FreeTrack is coded in Delphi 7 and uses DirectShow and DirectX. Head tracking is achieved using implementations of DeMenthon's four point iterative pose estimation algorithm (POSIT) and Alter's three point geometric algorithm.

Software

FreeTrack uses a camera to collect real-time information on the point model worn by the user. Specifically the image coordinates of the model points, which are either received directly from the camera or extracted from a video stream. These coordinates are used to generate an estimate of the real head pose, which can be transformed by the user in a number of ways to create a virtual pose. One of the most fundamental transformations involves amplifying rotation so that only small head movements are required. Finally, the virtual pose is sent to the user's choice of outputs. This is all done in the background, with tracking status displayed in the system tray.

A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first-person. This greatly assists with testing and makes it easier to experiment with different settings.

Each degree of freedom (axis) has a response curve that can be modified to change the way the virtual head moves for a given real head movement. This is commonly used to create a central deadzone region so that the user’s head can be more relaxed there.

Keyboard, mouse and joystick buttons can be used to toggle tracking settings, including the virtual centre location (like adjusting the seat position in a car) and individually toggle axes and outputs.

For NaturalPoint cameras, FreeTrack can provide advanced features and a level of customization that is not available with official software.

Camera

Comparison of some cameras compatible with FreeTrack

Camera	Sensor Resolution	Sub-Pixel Reporting Resolution	Sensor	FPS	Angle(°)	CPU usage	Approx. price (USD)	In-device Pre-processing
Cheap webcam	320x240		color	30	42	moderate	$20	no
Ideal webcam	640x480		monochrome	60+	42	small	?	no
Sony PlayStation EyeToy	640x480		color	30	56	?	$30	no
Sony PlayStation 3 Eye	320x240 or 640x480		color	120 or 60	56 or 75	?	$40	no
Microsoft Xbox Live Vision	640x480		color	30	?	?	$40	no
Nintendo Wii Remote	128x96	1024x768	monochrome	100	41	none	$40	yes
NaturalPoint TrackIR 1	60k pixels (eg 300x200)		monochrome	60	33	small	retired	yes
NaturalPoint TrackIR 2	60k pixels (eg 300x200)		monochrome	100	33	small	retired	yes
NaturalPoint TrackIR 3	355x288		monochrome	80	33	small	retired	yes
NaturalPoint TrackIR 3 Pro	355x288		monochrome	120	33	small	retired	yes
NaturalPoint TrackIR 4 Pro	355x288 (subsampled at 710x288)	7100 x 5760	monochrome	120	46	small	$99.95	yes
NaturalPoint TrackIR 5	640 x480	96000 x 72000	monochrome	120	51.7	?	$149.95	yes
NaturalPoint SmartNav 1/2	60k pixels (eg 300x200)		monochrome	60	33	small	retired	yes
NaturalPoint SmartNav 3	355x288		monochrome	120	33	small	retired	yes
NaturalPoint SmartNav 4	640x480 (subsampled at 1280x480)		monochrome	100	41	small	$400 to $500	yes

Resolution

In most cases a resolution of 320x240 is sufficient, this is capable of producing a much higher sub-pixel resolution, enough to allow accurate cursor control on a high resolution monitor. Resolutions 640x480 and above have diminishing returns and correspond with an exponential increase in CPU usage when not pre-processed before reaching the computer. Higher resolutions become more important at greater distances from the camera.

Sensor

For the same resolution, monochrome sensors can resolve finer detail much better than color sensors due to the lack of a color filter array.

Frame rate

FreeTrack uses interpolation with low frame rate video devices to improve panning smoothness. However, responsiveness is fundamentally limited to the frame rate; a 30frame/s webcam has a maximum response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120frame/s camera. To put this into perspective, a human’s reaction time to visual stimulus (finger reflex) is typically around 200ms, 30ms can be regarded as a competitive ping in online reflex-based games and an LCD monitor refresh rate is typically 17ms.

Higher responsiveness gives a greater feeling of control but since virtual head motion is amplified it can also cause it to move unrealistically fast. For this reason, some programs limit head movement speed, wasting some of the responsiveness of higher frame-rate cameras.

Angle

A wider viewing angle allows a larger tracking region when in close proximity to the camera. At further distances a wide angle is not desirable, more of the frame is unused and the effective resolution drops more rapidly. More peripheral light can also be seen which can interfere with tracking. Viewing angle can be reduced by using digital zoom at the cost of resolution.

CPU usage

The Nintendo Wii Remote effectively uses no CPU, NaturalPoint cameras use a small amount and general video devices can use a significant amount depending on the brand and the specific camera settings in use. A Playstation Eye running at the same resolution and frame rate as a TrackIR 4 would be very demanding on a single-core CPU. However, modern multi-core CPUs are making this less of an issue. Resolution and frame rate can always be reduced to conserve CPU resources.

Filters

For reasons of efficiency, FreeTrack requires the tracking points to be isolated from all other light, this is best done using infrared LEDs and a visible-light blocking filter in front of the camera. Photographic film or the magnetic storage medium inside floppy disks can be used as inexpensive visible-light filters. Further filtering can be done in software by adjusting exposure and threshold.

All video devices like webcams have a built-in infrared blocking filter which can be removed to improve sensitivity to infrared light, allowing better point isolation and the possibility of retroreflective tracking. This is normally a straight-forward and reversible procedure for most webcams.

Wii Remotes and NaturalPoint cameras are designed for infrared point tracking so already have visible-light blocking filters.

Point model

Model configurations

• Single point: a single tracking point, for example attached on a baseball cap or wire that extends forward of the head. Can be easily achieved with a small light source like a penlight. Unlike other configurations, this is limited to two degrees of freedom (yaw and pitch) and can be more sensitive to translation than rotation.
• Three point clip: asymmetrical triangular arrangement of points that can be clipped to headphones and positioned to the side of the head.
• Three point cap: symmetrical triangular arrangement of points, typically attached to the rim of a baseball cap.
• Four point cap: symmetrical non-coplanar arrangement of points, also attached to the rim of a baseball cap.

Models can be made in a DIY fashion at minimal expense using readily available electronic components. Component kits and fully constructed models are also available for purchase from some members of the FreeTrack community.

Active points

An active point model uses visible or infrared LEDs (5 mm or larger) to represent the tracking points, powered by battery, transformer (plug pack) or USB. The electric circuit is very basic and can be made by someone with little or no experience with electronics.

Common LEDs, like those found in remote controls, have a narrow, highly focused beam which is not suitable for optical motion tracking. They can be easily turned into wide angle LEDs by filing their lens tips down flat. Alternatively, wide angle LEDs can be purchased from specialist electronics retailers, like the infrared Siemens/Osram SFH485P, with a half-angle of 40 degrees.

Reflective points

Retroreflective material can be used to represent the tracking points by illumination with an infrared light source. This configuration doesn’t require wires or batteries connected to the user but is more susceptible to interference by background light. In most cases a webcam’s internal infrared blocking filter needs to be removed to increase sensitivity enough that the infrared light reflected by the tracking points can be seen.

FreeTrack interface

FreeTrack has a simple interface that can be freely used by third party programs to access 6DOF tracking data, both real raw measurements and virtual. It is hardware agnostic, so is not dependent on a specific brand or version of hardware and can be used without restriction.

TrackIR interface

The TrackIR interface consists of three different interfaces.

The original interface, which FreeTrack is compatible with, is used by most games released before October 2008.

A new version of the interface is used by some, but not all, games released after October 2008 with an encrypted data stream that is not backward compatible. FreeTrack is not compatible with this interface, so some games using it must be patched in order for FreeTrack to work with them. Some of the patches change the executable which prevents them from being used in online multi-player mode with games that include anti-cheating protection.

The third interface is a DLL software module called TIRViews developed by NaturalPoint which provides special support for a small number of games. The TIRViews module is distributed as part of the TrackIR software installer. FreeTrack is able to load the TIRViews module and gain view control in the titles it provides access to. However, doing so violates the license under which NaturalPoint distributes the TIRViews module. The license states "The TrackIR software product is composed of...and dll components", ""NaturalPoint...grants...license...to use the TrackIR software ONLY with NaturalPoint TrackIR Hardware"" and "Use of the TrackIR software with...anything which emulates a TrackIR is prohibited"

The list of FreeTrack compatible titles indicates which games use which interface.

The TrackIR interface is proprietary and is closed source.

TrackIR controversy

NaturalPoint Inc., the makers of TrackIR, believe that the use of FreeTrack to gain view control in TrackIR Enhanced software is a violation of their copyright.

FreeTrack is free, open source software. However, a file named "TIRServer.dcu" is only provided in the source repository in a compiled binary format, with no source available.

Most TrackIR Enhanced software need to be provided with text strings which bear notice of "EyeControl Technologies" copyright (former name of NaturalPoint, Inc.) in order to activate the TrackIR Enhanced interface. Software which requires these text strings for interface activation also contain the strings themselves. At NaturalPoint's request, FreeTrack project members removed the strings from the software they provide to end users. FreeTrack then implemented a workaround which creates a local copy of these strings from the client software when used with TrackIR Enhanced titles. FreeTrack project members argue that copyright is not violated in this case since it may fall under the provision of 17 U.S.C. § 117. The text strings are a necessary and functional part of the interface which FreeTrack project members believe makes them exempt from copyright for the purposes of interoperability.

NaturalPoint started using an encrypted data stream in version 4.1.036 of their TrackIR software, this made it more difficult for third party software like FreeTrack to interface with TrackIR Enhanced software titles. The first game to require the new data stream is DCS: Black Shark but a fix is available to make it compatible with FreeTrack.

See also

• List of games supporting FreeTrack
• TrackIR
• Cachya

External links

• Official website
• Original website
• Daniel DeMenthon's academic page - papers and examples of POSIT algorithm
• FreeTrack YouTube channel