Touchscreen

A touchscreen is a display which can detect the presence and location of a touch within the display area. The term generally refers to touch or contact to the display of the device by a finger or hand. Touchscreens can also sense other passive objects, such as a stylus. However, if the object sensed is active, as with a light pen, the term touchscreen is generally not applicable. The ability to interact directly with a display typically indicates the presence of a touchscreen.

The touchscreen has two main attributes. First, it enables one to interact with what is displayed directly on the screen, where it is displayed, rather than indirectly with a mouse or touchpad. Secondly, it lets one do so without requiring any intermediate device, again, such as a stylus that needs to be held in the hand. Such displays can be attached to computers or, as terminals, to networks. They also play a prominent role in the design of digital appliances such as the personal digital assistant (PDA), satellite navigation devices, mobile phones, and video games.

History

One of the Nintendo DS' main selling points is its second screen on the bottom, which is a touch screen.

Touchscreens emerged from academic and corporate research labs in the second half of the 1960s. One of the first places where they gained some visibility was in the terminal of a computer-assisted learning terminal that came out in 1972 as part of the PLATO project. They have subsequently become familiar in kiosk systems, such as in retail and tourist settings, on point of sale systems, on ATMs and on PDAs where a stylus is sometimes used to manipulate the GUI and to enter data. The popularity of smart phones, PDAs, portable game consoles and many types of information appliances is driving the demand for, and the acceptance of, touchscreens.

The HP-150 from 1983 was probably the world's earliest commercial touchscreen computer. It doesn't actually have a touchscreen in the strict sense, but a 9" Sony CRT surrounded by infrared transmitters and receivers which detect the position of any non-transparent object on the screen.

Until the early 1980s, most consumer touchscreens could only sense one point of contact at a time, and few have had the capability to sense how hard one is touching. This is starting to change with the commercialisation of multi-touch technology.

Touchscreens are popular in heavy industry and in other situations, such as museum displays or room automation, where keyboard and mouse systems do not allow a satisfactory, intuitive, rapid, or accurate interaction by the user with the display's content.

Historically, the touchscreen sensor and its accompanying controller-based firmware have been made available by a wide array of after-market system integrators and not by display, chip or motherboard manufacturers. With time, however, display manufacturers and System On Chip (SOC) manufacturers worldwide have acknowledged the trend toward acceptance of touchscreens as a highly desirable user interface component and have begun to integrate touchscreen functionality into the fundamental design of their products. New developments in this area include PSoC (Programmable System-on-Chip) by Cypress Semiconductor.

Technologies

There are a number of types of touchscreen technology.

Resistive

Main article: Resistive touchscreen

A resistive touchscreen panel is composed of several layers, the most important of which are two thin, metallic, electrically conductive layers separated by a narrow gap. When an object, such as a finger, presses down on a point on the panel's outer surface the two metallic layers become connected at that point: the panel then behaves as a pair of voltage dividers with connected outputs. This causes a change in the electrical current which is registered as a touch event and sent to the controller for processing.

Surface acoustic wave

Surface acoustic wave (SAW) technology uses ultrasonic waves that pass over the touchscreen panel. When the panel is touched, a portion of the wave is absorbed. This change in the ultrasonic waves registers the position of the touch event and sends this information to the controller for processing. Surface wave touchscreen panels can be damaged by outside elements. Contaminants on the surface can also interfere with the functionality of the touchscreen.

Capacitive

A capacitive touchscreen panel is a sensor typically made of glass coated with a material such as indium tin oxide. This type of sensor is basically a capacitor in which the plates are the overlapping areas between the horizontal and vertical axes in a grid pattern. Since the human body also conducts electricity, a touch on the surface of the sensor will affect the electric field and create a measurable change in the capacitance of the device. These sensors work on proximity, and do not have to be directly touched to be triggered. It is a durable technology that is used in a wide range of applications including point-of-sale systems, industrial controls, and public information kiosks. It has a higher clarity than Resistive technology, but it only responds to finger contact and will not work with a gloved hand or pen stylus unless the stylus is conductive. Capacitive touch screens can also support Multitouch. Examples include Apple Inc.'s iPhone and iPod touch, HTC's G1 & HTC Magic , Palm Inc.'s Palm Pre and Palm Eos and more recently the LG KM900 Arena, Microsoft's Zune HD, Sony Walkman X series and Sony Ericsson's Aino.

Projected capacitance

Projected Capacitance Touch technology is a type of capacitive technology which involves the relationship between an XY array of sensing wires embedded within two layers of non-metallic material, and a third object. In touchscreen applications the third object can be a human finger. Projected capacitance creates an electrostatic field above the sensing surface to determine inputs. This format requires the use of patterned ITO (Indium Tin Oxide) and requires no calibration. Capacitance forms between the user’s fingers and projected capacitance from the sensing wires. A touch is made, precisely measured, then passed on to the controller system which is connected to a computer running a software application. This will then calculate how the user’s touch relates to the computer software. Projected capacitive touchscreens enjoy the benefits of responding accurately to both fingers and stylis.

Visual Planet’s ViP Interactive Foil is an example of a product that uses Projected Capacitance Touch technology. This technology allows a gloved hand to make the touch, resulting in Projected Capacitance Touch technology now being common in external "through window" touch applications (i.e. those where no direct physical contact with the touchscreen is made).

Infrared

Conventional optical-touch systems use an array of infrared (IR) light-emitting diodes (LEDs) on two adjacent bezel edges of a display, with photosensors placed on the two opposite bezel edges to analyze the system and determine a touch event. The LED and photosensor pairs create a grid of light beams across the display. An object (such as a finger or pen) that touches the screen interrupts the light beams, causing a measured decrease in light at the corresponding photosensors. The measured photosensor outputs can be used to locate a touch-point coordinate.

Widespread adoption of infrared touch screens has been hampered by two factors: the relatively high cost of the technology compared to competing touch technologies and the issue of performance in bright ambient light. This latter problem is a result of background light increasing the noise floor at the optical sensor, sometimes to such a degree that the touch screen’s LED light cannot be detected at all, causing a temporary failure of the touch screen. This is most pronounced in direct sunlight conditions where the sun has a very high energy distribution in the infrared region.

However, certain features of infrared touch remain desirable and represent attributes of the ideal touch screen, including the option to eliminate the glass or plastic overlay that most other touch technologies require in front of the display. In many cases, this overlay is coated with an electrically conducting transparent material such as indium tin oxide (ITO), which reduces the optical quality of the display. This advantage of optical touch screens is extremely important for many device and display vendors since devices are often sold on the perceived quality of the user display experience.

Another feature of infrared touch which has been long desired is the digital nature of the sensor output when compared to many other touch systems that rely on analog-signal processing to determine a touch position. These competing analog systems normally require continual re-calibration, have complex signal-processing demands (which adds cost and power consumption), demonstrate reduced accuracy and precision compared to a digital system, and have longer-term system-failure modes due to the operating environment.

Finally, infrared touch is not capable of multi-touch as a result of the mode of operation being one in which lines of light are disrupted. Thus for two points, two beams would be disrupted on both the horizontal and vertical axes each, allowing for a total of four potential point locations. This ambiguity makes infrared unsuitable for multi-touch applications.

Neonode has taken conventional infrared touch technology, using LEDs and photodiodes, and essentially miniaturized it and reduced the cost for use in handheld devices. In addition to using the technology in its own N2 cell phone, Neonode is also marketing it to other device makers.

Strain gauge

In a strain gauge configuration, also called force panel technology, the screen is spring-mounted on the four corners and strain gauges are used to determine deflection when the screen is touched. This technology has been around since the 1960s but new advances by Vissumo and F-Origin have made the solution commercially viable. It can also measure the Z-axis and the force of a person's touch. Typically used in exposed public systems such as ticket machines due to their resistance to vandalism.

Optical imaging

A relatively-modern development in touchscreen technology, two or more image sensors are placed around the edges (mostly the corners) of the screen. Infrared backlights are placed in the camera's field of view on the other sides of the screen. A touch shows up as a shadow and each pair of cameras can then be triangulated to locate the touch or even measure the size of the touching object (see visual hull). This technology is growing in popularity, due to its scalability, versatility, and affordability, especially for larger units.

Dispersive signal technology

Introduced in 2002 by 3M, this system uses sensors to detect the mechanical energy in the glass that occurs due to a touch. Complex algorithms then interpret this information and provide the actual location of the touch. The technology claims to be unaffected by dust and other outside elements, including scratches. Since there is no need for additional elements on screen, it also claims to provide excellent optical clarity. Also, since mechanical vibrations are used to detect a touch event, any object can be used to generate these events, including fingers and stylus. A downside is that after the initial touch the system cannot detect a motionless finger.

Acoustic pulse recognition

This system, developed by Tyco International's Elo division, uses more than two piezoelectric transducers located at some positions of the screen to turn the mechanical energy of a touch (vibration) into an electronic signal. The screen hardware then uses an algorithm to determine the location of the touch based on the transducer signals. This process is similar to triangulation used in GPS. The touchscreen itself is made of ordinary glass, giving it good durability and optical clarity. It is usually able to function with scratches and dust on the screen with good accuracy. The technology is also well suited to displays that are physically larger. As with the Dispersive Signal Technology system, after the initial touch, a motionless finger cannot be detected.

Building touch screens

There are several principal ways to build a touch screen. The key goals are to recognize one or more fingers touching a display, to interpret the command that this represents, and to communicate the command to the appropriate application.

In the most popular techniques called the capacitive or resistive approach, manufactures coat the screen with a thin, transparent metallic layer. When a user touches the surface, the system records the change in the electrical current that flows through the display.

Dispersive-signal technology which 3M created in 2002, measures the piezoelectric effect- the voltage generated when mechanical force is applied to a material- that occurs chemically when a strengthened glass substrate is touched.

There are two infrared-based approaches. In one, any array of sensors detects finger touching or almost touching the display, there by interrupting light beams projected over the screen. In the other, bottom-mounted infrared cameras record screen touches.

In each case, the system determines the intended command based on the controls showing on the screen at the time and the location of the touch.

Development

Virtually all of the significant touchscreen technology patents were filed during the 1970s and 1980s and have expired. Touchscreen component manufacturing and product design are no longer encumbered by royalties or legalities with regard to patents and the manufacturing of touchscreen-enabled displays on all kinds of devices is widespread.

The development of multipoint touch screens facilitated the tracking of more than one finger on the screen, thus operations that require more than one finger are possible. These devices also allow multiple users to interact with the touchscreen simultaneously.

With the growing acceptance of many kinds of products with an integral touchscreen interface the marginal cost of touchscreen technology is routinely absorbed into the products that incorporate it and is effectively eliminated. As typically occurs with any technology, touchscreen hardware and software has sufficiently matured and been perfected over more than three decades to the point where its reliability is unassailable. As such, touchscreen displays are found today in airplanes, automobiles, gaming consoles, machine control systems, appliances and handheld display devices of every kind. With the influence of the multi touch-enabled iPhone and the Nintendo DS, the touchscreen market for mobile devices is projected to produce US$5 billion in 2009.

The ability to accurately point on the screen itself is taking yet another step with the emerging graphics tablet/screen hybrids.

Ergonomics and usage

Finger stress

An ergonomic problem of touchscreens is their stress on human fingers when used for more than a few minutes at a time, since significant pressure can be required for certain types of touchscreen. This can be alleviated for some users with the use of a pen or other device to add leverage and more accurate pointing. However, the introduction of such items can sometimes be problematic depending on the desired use case (for example, public kiosks such as ATMs). Also, fine motor control is better achieved with a stylus, because a finger is a rather broad and ambiguous point of contact with the screen.

Fingernail as stylus

These ergonomic issues of direct touch can be bypassed by using a different technique, provided that the user's fingernails are either short or sufficiently long. Rather than pressing with the soft skin of an outstretched fingertip, the finger is curled over, so that the top of the forward edge of a fingernail can be used instead. (The thumb is optionally used to provide support for the finger or for a long fingernail, from underneath.)

The fingernail's hard, curved surface contacts the touchscreen at a single very small point. Therefore, much less finger pressure is needed, much greater precision is possible (approaching that of a stylus, with a little experience), much less skin oil is smeared onto the screen, and the fingernail can be silently moved across the screen with very little resistance, allowing for selecting text, moving windows, or drawing lines.

The human fingernail consists of keratin which has a hardness and smoothness similar to the tip of a stylus (and so will not typically scratch a touchscreen). Alternately, very short stylus tips are available, which slip right onto the end of a finger; this increases visibility of the contact point with the screen. Oddly, with capacitive touchscreens, the reverse problem applies in that individuals with long nails have reported problems getting adequate skin contact with the screen to register keystrokes (note that ordinary styli do not work on capacitive touchscreens nor do gloved fingers).

The concept of using a fingernail trimmed to form a point, to be specifically used as a stylus on a writing tablet for communication, appeared in the 1950 science fiction short story Scanners Live in Vain.

Fingerprints

Touch screens can suffer from the problem of fingerprints on the display. This can be mitigated by the use of materials with optical coatings designed to reduced the visible effects of fingerprint oils, or by reducing skin contact by using a fingernail or stylus instead.

Combined with haptics

The user experience with touchscreens without tactile feedback or haptics can be difficult due to latency or other factors. Research from the University of Glasgow Scotland [Brewster, Chohan, and Brown 2007] demonstrates that sample users reduce input errors (20%), increase input speed (20%), and lower their cognitive load (40%) when touch screens are combined with haptics or tactile feedback, [vs. non-haptic touch screens].

"Gorilla arm"

Gorilla arm was a side-effect that destroyed vertically-oriented touch-screens as a mainstream input technology despite a promising start in the early 1980s. Designers of touch-menu systems failed to notice that humans are not built to hold their arms at waist- or head-height, making small and precise motions. After a short period of time, cramp may begin to set in, and arm movement becomes painful and clumsy. This is now considered a classic cautionary tale to human-factors designers; "Remember the gorilla arm!" is shorthand for "How is this going to fly in real use?". Gorilla arm is not a problem for specialist short-term-use devices such as ATMs, since they only involve brief interactions which are not long enough to cause gorilla arm. Gorilla arm also can be mitigated by the use of horizontally-mounted screens such as those used in Tablet PCs, but these need to account for the user's need to rest their hands on the device. This can increase the amount of dirt deposited on the device, and occludes the user's view of the screen.

See also

• Energy harvesting
• Flexible keyboard
• Gestural interface
• Graphics tablet
• Graphics tablet-screen hybrid
• Light Pen
• Multi-touch
• Smartphone
• Tablet PC
• Touchpad
• Touch switch

References

This article is based on material taken from the Free On-line Dictionary of Computing prior to 1 November 2008 and incorporated under the "relicensing" terms of the GFDL, version 1.3 or later.

• Andreas Holzinger: Finger Instead of Mouse: Touch Screens as a means of enhancing Universal Access, In: Carbonell, N.; Stephanidis C. (Eds): Universal Access, Theoretical Perspectives, Practice, and Experience. Lecture Notes in Computer Science. Vol. 2615. Berlin, Heidelberg, New York: Springer, 2003, ISBN 3-540-00855-1, 387–397.

External links

• Touch Screen Comparison - GTOUCH
• Howstuffworks - How do touchscreen monitors know where you're touching?
• MERL - Mitsubishi Electric Research Lab (MERL)'s research on interaction with touch tables.
• Jefferson Y. Han et al. Multi-Touch Interaction Research. Multi-Input Touchscreen using Frustrated Total Internal Reflection.
• Dot-to-Dot Programming : Building Microcontrollers
• EDN 11/9/95 - A great, but old, article that gets into some nice specifics.
• Typical SAW Touchscreen parameters