Multi-touch

Multi-touch screen

Multi-touch (or multitouch) denotes a set of interaction techniques which allow computer or mobile users to control graphical user interface with more than one finger at either application or system level.

Multi-touch consists of a touch screen (screen,overlay, table, wall, etc.) or touchpad, as well as software that recognizes multiple simultaneous touch points, as opposed to the single touchscreen (e.g. computer touchpad, ATM), which recognizes only one touch point. This effect is achieved through a variety of means, including but not limited to: heat, finger pressure, high capture rate cameras, infrared light, optic capture, tuned electromagnetic induction, ultrasonic receivers, transducer microphones, laser rangefinders, and shadow capture.

Many products for multi-touch interfaces exist and are being proposed. Multi-touch is used on portable devices including Apple Inc's iPhone and Palm Pre as well as other desktop products Microsoft Surface and the DELL Latitude XT2.

History

Multi-touch technology dates back to 1982, when Nimish Mehta at the University of Toronto developed the first finger pressure multi-touch display.

Bell Labs

In 1983, Bell Labs at Murray Hill published a comprehensive discussion of touch-screen based interfaces.

In 1984 Bell Labs engineered a touch screen that could change images with more than one hand. The group at the University of Toronto stopped working on hardware and moved on to software and interfaces, expecting that they would have access to the Bell Labs work.

A breakthrough occurred in 1991, when Pierre Wellner published a paper on his multi-touch “Digital Desk”, which supported multi-finger and pinching motions.

Recent developments

Various companies expanded upon these discoveries in the beginning of the twenty-first century. Mainstream exposure to multi-touch technology occurred in the year 2007, when Apple unveiled the iPhone and Microsoft debuted surface computing. The iPhone in particular has spawned a wave of interest in multi-touch computing, since it permits greatly increased user interaction on a small scale. More robust and customizable multi-touch and gesture-based solutions are beginning to become available, among them TrueTouch, created by Cypress Semiconductor. The following is a compilation of notable uses of multi-touch technology in recent years.

The future of multi-touch

The use of multi-touch technology is expected to rapidly become common place. For example, touch screen telephones are expected to increase from 200,000 shipped in 2006, to 21 million in 2012.

Multi-touch Displays

Perceptive Pixel

Jefferson Y. Han debuting his initial multi-touch system at TED

Perceptive Pixel is a company founded by New York University consulting research scientist Jefferson Y. Han that creates wall displays and tables. The displays use infrared light emitting diodes along with an infrared camera to determine the point of contact. Han envisions large collaborative spaces that will allow multiple users to work and interact. Perceptive Pixel’s technology is currently being utilized, in the form of the Multi-Touch Collaboration Wall, by CNN and an unspecified government contractor. But its 6-figer price even scared most interested medium business away.

Microsoft Surface

Microsoft Surface.

Main article: Microsoft Surface

In 2001 Steve Bathiche and Andy Wilson of Microsoft began work on an idea for an interactive table that mixes both physical and virtual worlds. Research and Development expanded rapidly in 2004, once the idea caught the attention of Microsoft Chairman Bill Gates. In 2007 Microsoft introduced Microsoft Surface, a functional multi-touch table-top computer based on a standard PC platform including an Intel Core 2 Duo processor, Windows Vista, and 2 GB of RAM.

Essentially, Microsoft Surface is a computer embedded in a table with a large, flat, touch-responsive display on the top. The table uses small cameras (as opposed to finger pressure or heat) that enable it to react to the touch of any object. The unit has eight different modes that allow users to perform an array of activities,ranging from organizing pictures and videos to ordering food at a restaurant. Multiple users have the ability to work on the table at one time. The preliminary launch was on April 17, 2008, when Surface became available for customer use in AT&T stores. Although the product is not available for widespread purchase, it can possibly be bought directly from Microsoft for $13,500.

3M Multi-touch

3M is a company known for many innovations in the touch field, including overlays incorporated in other vendor's products. 3M provides a Multi-touch developer kit which includes a projected capacitive glass surfaced 19" monitor with stand and a software development kit which includes support for the upcoming Windows 7 operating system. The unique features at this price point are 10 finger multi-touch and 15ms touch response with full edge support and finger identification when both are touching.

Tyco Electronics: Elo Touch Systems

Tyco Electronics is a electronics component manufacturer which also sells touch technology through their Elo Touch Systems division. Elo Touch Systems provides integrator components, custom solutions, kits, and fully integrated touch monitors through resellers. Elo Touch Systems provides multiple technologies for single touch screens: Acoustic Pulse Recognition (APR); IntelliTouch Surface Acoustic Wave (SAW); and CarrollTouch InfraRed (IR). For Multi-touch they have been an early integrator working on Microsoft Windows 7, initially announcing support for 2 finger multi-touch support with IntelliTouch Surface Acoustic Wave and technology built on CarrollTouch IR touch technologies, in both 19" and 22" form factors. .

Demand Evolution

Demand Evolution has released two multi-touch displays based on infrared vision sensing and open-source software. Its product, the LCD based Gecko, is one of the cross platform, high-resolution multitouch displays available directly for consumers. The interactive display utilizes a 30 inch LCD, and as opposed to the wall mount and table designs of many competitors the Gecko is designed for workstation and kiosk scenarios. Currently, such open source multi-touch units on the market are low cost, because no developer kit, business software, or OS can directly support. It is aimed only at the open-source developer community and early adopters hoping to get in early on the industry shift to multi-touch interaction.

MultiTouch LCD Cell

MultiTouch LCD Cell.

In September 2008,MultiTouch Oy launched a large-scale LCD-based multitouch display, the MultiTouch Cell. This model was a 46-inch display with full-HD LCD panel. The central point of the LCD-based display is to handle the shortcomings of projector-based displays (poor contrast and resolution, frequent maintenance needs).

HP TouchSmart

HP Touchsmart is an All-In-One PC introduced by computer giant Hewlett-Packard and was first released in 2007. It is an example of the PC-In-A-Box terminology quite similar to the iMac G3 except that it utilizes more modern concepts such as better performing hardware, Windows Vista OS and of course, its 19"+ flat panel multi-touch display. It is the first mass marketed 'touch screen' PC made commercially available. The technology behind the first generation TouchSmart, however, does not support the same class of multi-touch interaction as the previously mentioned products. Because of the nature of its optical touch sensors, the display can not effectively sense touches that fall within the same sensing beam. This shadowing problem makes the screens prohibitive for many of the multi-touch interaction scenarios that other options can support. Newer generation TouchSmart systems feature larger display sizes, up to 25.5", and improved touch technology. The improvements confine the "area of confusion" to a smaller footprint, inhibiting multi-touch by very small fingers immediately adjacent to each other.

Multi-touch Pads

Fingerworks

In 1999, Fingerworks, a Newark-based company run by University of Delaware academics John Elias and Wayne Westerman, produced a line of multi-touch products including the iGesture Pad and the TouchStream keyboard. Westerman published a dissertation in 1999 on the subject. In 2005, after years of maintaining a niche line of keyboards and touchpads, Fingerworks was acquired by Apple Inc.

Apple products

In 2005, Apple acquired Fingerworks. In 2007 they introduced the iPhone, marking the first time multi-touch technology was used on a phone. The iPhone includes such components as a web browser, music player, video player, and a cell phone without the use of a hard keypad or stylus.

Following the release of the iPhone, Apple also expanded its use of multi-touch computing with the new iPod Touch, as well as the new MacBook Air. As of 2008, Multi-touch can be found on the MacBook and MacBook Pro line in the form of a trackpad.

The latest revisions of Apple's Unibody MacBook and MacBook Pro features a full glass multi-touch trackpad (whilst the MacBook Air features a standard multi-touch trackpad). These enable various gestures such as scrolling, "swiping" between pages or pictures as well as rotating pictures, and launching Expose.

CUBIT and TouchKit

Addie Wagenknecht and Stefan Hechenberger of Nortd studio, based in New York and Vienna, created a scaled-down open-source version of Surface, called CUBIT and a screen based kit called TouchKit. According to the website of Technology Review magazine, "By sharing the Cubit's hardware schematics and software source code, the engineers are significantly reducing the cost of owning a multitouch table. But they're also fostering innovation by giving engineers an open platform on which to develop novel multitouch applications--something that they've previously lacked. It will cost anywhere between $1,000 to $1,580 depending on hardware used."

Asus EEE PC

Asus has included a multi-touch touchpad in the 900 and 1000 series of their Eee PC netbooks, an Elantech Smart-Pad. Depending on the installed drivers, it can be used for scrolling, 3 button operation, dragging and resizing and rotating pictures.

Dell Inspiron Mini

Dell has also included a multi-touch touchpad in the Dell Inspiron Mini Series. Provided with a driver from Elantech, it can be used for scrolling, 3 button operation, dragging, resizing, rotating pictures and system shortcuts as Start Menu and Explorer (on Windows), browser back and forward, minimizing and maximizing windows, window switching, desktop showing and hiding, and running custom commands.

Dell Latitude XT

In July 2008, Dell released multi-touch touch-screen drivers for the Latitude XT Tablet, claiming the "industry’s first convertible tablet with multi-touch capabilities." Dell has partnered with multi-touch technology startup N-trig to enable multi-touch capabilities for its tablet.

There have been numerous reports of serious problems with the XT multi-touch drivers or hardware interface, which, as of October 2008, remain unresolved by Dell. The major issue is centered around the multi-touch drivers failing when the computer is booted when USB devices are connected at boot, or when certain internal media drives are present. Often, but not always, the issue is resolved if the computer is booted without the USB devices and they are connected after boot. A similar, but not identical, issue may also be caused by interfering software, such as the iTunes helper. Issue resolution is complicated by the possible failure of the driver install program to function in the presence of damaged drivers, requiring manual removal of related files and registry keys.. See External Links for links to user discussions of this issue. With the introduction of Windows 7 which offers native multi-touch support, preliminary tests have show that many of these issues with the XT have been solved, together with a much improved N-Trig Win 7 driver.

Sharp Mobius

In April 2009, Sharp unveiled the first laptop with an optical-sensor LCD pad. The optical-sensor LCD technology combines LCD and CCD elements within its pixels and can detect fingers and objects such as a stylus, and is claimed to be able to scan a business card placed on top of the screen. Further improvements to this function are expected to enable fingerprint authentication in the future.

Lenovo Tablets (x200)

Currently, Lenovo's definition of "Multitouch" implies that the screen responds to either a finger or a digitizer pen. However, it is not capable of detecting simultaneous touch points on the tablet screen nor any forms of multi-finger gestures, while the Windows 7 already supports simple dual-touch ones.

Software

Mac OS X Tiger

Apple Mac OS X 10.4, as well as derivatives such as iPhone OS support multi-touch. This extends from the windowing environment, through to Safari, iLife and other software.

Linux

Various versions of Linux, and especially those versions used on small devices such as the Asus Eee PC and Palm Pre, have multi-touch support.

Windows 7

File:Windows 7 Touch Photos Application.jpg

A multi-touch photos application is demonstrated in Windows 7.

Windows 7 supports Multi-touch. The operating system is known to have a multi-touch mapping application, photo viewing program, and incorporation in Internet Explorer 8. In January 2009, Microsoft joined with other investors who invested twenty-four million dollars in N-Trig Ltd., which plans to make computer hardware that takes advantage of Windows 7's multi-touch support..

22MILES

One of the advantages of 22MILES multi-touch framework is OS and multi-touch hardware independent. Their multi-touch software can run Windows 7, XP, Vista, Ubuntu Linux and Mac OS X, while it supports various multi-touch hardware technology, such as Infrared Matrix/Camera, Projected Capacitive, SAW (dual-touch), resistive, FTIR/ DI/ DSI, LED-LP. But unlike open-source, their proprietary software and multitouch gestures only target to business, such as digital signage, Medical Industry, GIS, Real Estate, and so on.

Popular culture references

Pop culture has also portrayed potential uses of multi-touch technology in the future. The 2002 science-fiction film Minority Report showed different Multi-touch features like enlarging and moving objects around. The television series CSI: Miami introduced both surface and wall displays similar to Microsoft Surface in its sixth season. Another form of a multi-touch computer was seen in the motion picture starring Ewan McGregor The Island, where the professor played by Sean Bean has a multi-touch desktop to organize files, which was based on an early version of Microsoft Surface^[8]. This technology can also be seen in the James Bond film Quantum of Solace. In a parodic episode of the popular TV series The Simpsons, when Lisa Simpson travels to the underwater headquarters of Mapple to visit Steve Mobs, the erstwhile pretender to the throne of Apple is shown to be performing multiple multi-touch hand gestures on a large touch wall.

A device similar to the Surface was seen in the 1982 movie Tron. It took up an executive's entire desk and was used to communicate with the Master Control computer.

See also

Multi-Touch Conception

• Human-Computer Interaction
• Surface Computing

Multi-Touch Mobile Devices

• iPhone & iPod Touch
• Palm Pre
• Zune HD
• Meizu M8
• Creative Zii
• Cypress TrueTouch
• Haptics
• Lemur Input Device

Multi-Touch Screen

• Jefferson Y. Han (and his company, Perceptive Pixel)
• Microsoft Surface
• TouchLight
• reactable
• SmartSurface
• Multitouch.fi
• CUBIT
• MPX

External links

• Multi-Touch Systems that I Have Known and Loved - An overview by researcher Bill Buxton of Microsoft Research, formerly at University of Toronto and Xerox PARC.
• The Unknown History of Pen Computing contains a history of pen computing, including touch and gesture technology, from approximately 1917 to 1992.
• Annotated bibliography of references to pen computing
• Multi-touch in Windows 7
• Multi-Touch Interaction Research @ NYU
• Jeff Han narrated video at TED conference
• Camera-based multi-touch for wall-sized displays
• David Wessel Multitouch