Chroma key
Chroma key is a technique for mixing two images or frames together, in which a color (or a small color range) from one image is removed (or made transparent), revealing another image behind it. This technique is also referred to as color keying, colour-separation overlay (CSO; primarily by the BBC[1]), greenscreen, and bluescreen. It is commonly used for weather forecast broadcasts, wherein the presenter appears to be standing in front of a large map, but in the studio it is actually a large blue or green background. The meteorologist stands in front of a bluescreen, and then different weather maps are added on those parts in the image where the color is blue. If the meteorologist himself wears blue clothes, his clothes will become replaced with the background video. This also works for greenscreens, since blue and green are considered the colors least like skin tone. This technique is also used in the entertainment industry, the iconic theatre shots in Mystery Science Theater 3000, for example.
History
Prior to the introduction of digital compositing, the process was a complex and time consuming one known as "travelling matte". The blue screen and traveling matte method were developed in the 1930s at RKO Radio Pictures and other studios, and were used to create special effects for The Thief of Bagdad (1940). At RKO, Linwood Dunn used travelling matte to create "wipes" – where there were transitions like a windshield wiper in films such as Flying Down to Rio (1933).
The credit for development of the bluescreen is given to Larry Butler, who won the Academy Award for special effects for The Thief of Bagdad. He had invented the blue screen and traveling matte technique in order to achieve the visual effects which were unprecedented in 1940. He was also the first special effects man to have created these effects in Technicolor, which was in its infancy at the time.
In 1950, Warner Brothers employee and ex-Kodak researcher Arthur Widmer began working on an ultra violet traveling matte process. He also began developing bluescreen techniques: one of the first films to use them was the 1958 adaptation of the Ernest Hemingway novella, The Old Man and the Sea, starring Spencer Tracy.[2]
The background footage was shot first and the actor or model was filmed against a bluescreen carrying out their actions. To simply place the foreground shot over the background shot would create a ghostly image over a blue-tinged background. The actor or model must be separated from the background and placed into a specially-made "hole" in the background footage. The bluescreen shot was first rephotographed through a blue filter so that only the background is exposed. A special film is used that creates a black and white negative image — a black background with a subject-shaped hole in the middle. This is called a 'female matte'. The bluescreen shot was then rephotographed again, this time through a red and green filter so that only the foreground image was cast on film, creating a black silhouette on an unexposed (clear) background. This is called a 'male matte'.
The background image is then rephotographed through the male matte, and the shot rephotographed through the female matte. An optical printer with two projectors, a film camera and a 'beam splitter' combines the images together one frame at a time. This part of the process must be very carefully controlled to ensure the absence of 'black lines'. During the 1980s, minicomputers were used to control the optical printer. For The Empire Strikes Back, Richard Edlund created a 'quad optical printer' that accelerated the process considerably and saved money. He received a special Academy Award for his innovation.
One drawback to the traditional traveling matte is that the cameras shooting the images to be composited can't be easily synchronized. For decades, such matte shots had to be done "locked-down" so that neither the matted subject nor the background could shift their camera perspective at all. Later, computer-timed motion control cameras alleviated this problem, as both the foreground and background could be filmed with the same camera moves.
Petro Vlahos was awarded an Academy Award for his development of these techniques. His technique exploits the fact that most objects in real-world scenes have a color whose blue color component is similar in intensity to their green color component. Zbig Rybczynski also contributed to bluescreen technology.
For Star Trek: The Next Generation, an ultraviolet light matting process was proposed by Don Lee of CIS and developed by Gary Hutzel and the staff of Image G. This involved a fluorescent orange backdrop which made it easier to generate a holdout matte, thus allowing the effects team to produce effects in a quarter of the time needed for other methods.[3]
Some films make heavy use of chroma key to add backgrounds that are constructed entirely using computer-generated imagery (CGI). Performances from different takes can even be composited together, which allows actors to be filmed separately and then placed together in the same scene. Chroma key allows performers to appear to be in any location without even leaving the studio.
Computer development also made it easier to incorporate motion into composited shots, even when using handheld cameras. Reference-points can now be placed onto the colored background (usually as a painted grid, X's marked with tape, or equally spaced tennis balls attached to the wall). In post-production, a computer can use the references to adjust the position of the background, making it match the movement of the foreground perfectly. Modern advances in software and computational power have even eliminated the need to use grids or tracking marks – the software analyzes the relative motion of colored pixels against other colored pixels and solves the 'motion' to create a camera motion algorithm which can be used in compositing software to match the motion of composited elements to a moving background plate.
Weathermen often use a field monitor to the side of the screen to see where they are putting their hands. A newer technique is to project a faint image onto the screen.
The process
The principal subject is filmed or photographed against a background consisting of a single color or a relatively narrow range of colors, usually blue or green because these colors are considered to be the furthest away from skin tone. The portions of the video which match the preselected color are replaced by the alternate background video. This process is commonly known as "keying", "keying out" or simply a "key".
Green is currently used as a backdrop more than any other color because image sensors in cameras are most sensitive to green. Therefore the green camera channel contains the least "noise" and can produce the cleanest key/matte/mask. Additionally, less light is needed to illuminate green because of the higher sensitivity to green in the image sensors.[4] Blue was used before digital keying became commonplace because it was necessary for the optical process, but it needed more illumination than green. Bright green has also become favored as a blue background may match a subject's eye color.
In analog color TV, color is represented by the phase of the chroma subcarrier relative to a reference oscillator. Chroma key is achieved by comparing the phase of the video to the phase corresponding to the preselected color. In-phase portions of the video are replaced by the alternate background video.
In digital color TV, color is represented by three numbers (red, green, blue). Chroma key is achieved by a simple numerical comparison between the video and the preselected color. If the color at a particular point on the screen matches (either exactly, or in a range), then the video at that point is replaced by the alternate background video.
Clothing
A chroma key subject must not wear clothing similar in color to the chroma key color(s) (unless intentional), because the clothing may be replaced with the background video. An example of intentional use of this is when an actor wears a blue covering over a part of his body to make it invisible in the final shot. This technique can be used to achieve an effect similar to that used in the Harry Potter films to create the effect of an invisibility cloak.[5]
In the 2002 film Spider-Man, in scenes where both Spider-Man and Green Goblin are in the air, Spider-Man had to be shot in front of the greenscreen & the goblin had to be shot in front of a bluescreen, because Spider-Man wears a costume whch is red and blue in color and the goblin wears a costume which is entirely green in color. If both were shot in front of same screen, one character would have been erased from the shot.
Background
Blue is generally used for both weather maps and special effects because it is complementary to human skin tone. The use of blue is also tied to the fact that the blue emulsion layer of film has the finest crystals and thus good detail and minimal grain (in comparison to the red and green layers of the emulsion.) In the digital world, however green has become the favored color because digital cameras retain more detail in the green channel and it requires less light than blue. Green not only has a higher luminance value than blue but also in early digital formats the green channel was sampled twice as often as the blue, making it easier to work with. The choice of color is up to the effects artists and the needs of the specific shot. In the past decade, the use of green has become dominant in film special effects. Also, the green background is favored over blue for outdoors filming where the blue sky might appear in the frame and could accidentally be replaced in the process. Although green and blue are the most common, any color can be used. Red is usually avoided due to its prevalence in normal human skin pigments, but can be often used for objects and scenes which do not involve people.
Occasionally, a magenta background is used, as in some software applications where the magenta or fuchsia key value #FF00FF is sometimes referred to as "magic pink".[6]
With better imaging and hardware, many companies are avoiding the confusion often experienced by weather presenters, who must otherwise watch themselves on a monitor to see the image shown behind them, by lightly projecting a copy of the background image onto the blue/green screen. This allows the presenter to accurately point and look at the map without referring to monitors.
A newer technique is to use a retroreflective curtain in the background, along with a ring of bright LEDs around the camera lens. This requires no light to shine on the background other than the LEDs, which use an extremely small amount of power and space unlike big stage lights, and require no rigging. This advance was made possible by the invention of practical blue LEDs in the 1990s, which also allow for emerald green LEDs.
There is also a form of color keying that uses light spectrum invisible to human eye. Called Thermo-Key, it uses infrared as the key color, which would not be replaced by background image during postprocessing.[7][8]
Even lighting
The biggest challenge when setting up a bluescreen or greenscreen is even lighting and the avoidance of shadow, because it is best to have as narrow a color range as possible being replaced. A shadow would present itself as a darker color to the camera and might not register for replacement. This can sometimes be seen in low-budget or live broadcasts where the errors cannot be manually repaired. The material being used affects the quality and ease of having it evenly lit. Materials which are shiny will be far less successful than those that are not. A shiny surface will have areas that reflect the lights making them appear pale, while other areas may be darkened. A matte surface will diffuse the reflected light and have a more even color range. In order to get the cleanest key from shooting greenscreen it is necessary to create a value difference between the subject and the greenscreen. In order to differentiate the subject and screen a two-stop difference can be used, either by making the greenscreen two stops higher than the subject or vice versa.
Programming
There are several different quality- and speed-optimized techniques for implementing color keying in software.[9]
In most versions, a function f(r,g,b)->a is applied to every pixel in the image. a <= 0 means the pixel is the green screen, a >= 1 means the pixel is in the foreground object. Values between 0 and 1 indicate a pixel that is partially covered by the foreground object. A usable green screen example, which matches how chroma key was done on an optical printer, is f(r,g,b)->K0*b-K1*g+K2 (K0..2 are user-adjustable constants, 1 is a good initial guess for all of them).
Often the software does screen spill removal from the colors as well as figure out the alpha. This may be a separate function g(r,g,b)->(r,g,b), a very simple green screen example is g(r,g,b)->(r,min(g,b),b). Or f is changed to return (r,g,b,a) all at once, this is useful if part of the calculation is shared.
Most keyers use far more complicated functions. A popular approach is to describe a closed 3D surface in RGB space and determine the signed distance the point (r,g,b) is from this surface, or to find the distance the point (r,g,b) is between two closed nested surfaces. It is also very common for f() to depend on more than just the current pixel's color, it may also use the x,y position, the values of nearby pixels, the value from reference images, and values from user-drawn masks.
A different class of algorithm tries to figure out a 2D path that separates the foreground from the background. This path can be the output, or the image can be drawn by filling the path with alpha=1 as a final step. An example of such an algorithm is the use of active contour. Most research in recent years has been into these algorithms.
See also
- Compositing
- Federal Standard 1037C
- Drew Carey's Green Screen Show
- Film production
- Front projection effect
- Matte (filmmaking)
- Primatte chromakey technology
- Optical printer
- Rear projection effect
- Reverse bluescreen
- Schüfftan process
- Signal processing
- Sodium vapor process
- Video
- Virtual set
References
- ^ "Glossary: Post Production". The Production Guide. Retrieved 2009-01-21.
- ^ "Illusions Take Home First Oscars". CRI English. 2005-02-14. Retrieved 2009-01-21.
- ^ Sternbach, Rick; Okuda, Michael (1991). Star Trek: The Next Generation Technical Manual. Pocket Books. p. 13. ISBN 0-671-70427-3.
- ^ http://www.destudiodublin.com/Facilities/Greenscreen-deStudio.htm
- ^ Creating an invisible cape in After Effects
- ^ "So you wanna make a theme?". skinyourscreen.com articles. Retrieved 2008-08-23.
- ^ "What is Thermo-Key?". University of Tokyo. Retrieved 2009-01-21.
- ^ Yasuda, K. "Thermo-key: human region segmentation from video". IEEE. Retrieved 2009-01-21.
{{cite web}}: Unknown parameter|coauthors=ignored (|author=suggested) (help) - ^ Ashihkmin, Michael. "High Quality Chroma Key".