Jump to content

Path tracing

From Wikipedia, the free encyclopedia
An image rendered using path tracing, demonstrating notable features of the technique
3D rendered image showing three copies of a cartoon cow. The one on the left has a metallic surface, and the one on the right uses a transparent glass material. The cow in the center appears made of glazed porcelain. The cows are standing on a wooden table. Lights and other background details from a cafe environment are reflected in the slightly glossy table and the cows.
A photorealistic path traced image, using Blender's Cycles renderer with image-based lighting

Path tracing is a rendering algorithm in computer graphics that simulates how light interacts with objects and participating media to generate realistic (physically plausible) images. It is based on earlier, more limited, ray tracing algorithms.

Path tracing is used to create photorealistic images for artistic purposes, and for applications such as architectural rendering and product design. It is also used to render frames for animated films, and visual effects for film and television. Because it can be very accurate and unbiased, it is commonly used to generate reference images when testing the quality of other rendering algorithms.

The technique uses the Monte Carlo method to compute estimates of global illumination and simulate the ways different materials reflect (or scatter), transmit, absorb, and emit light. It can incorporate simple modeling of the effects of aperture and lens (depth of field, and bokeh) and shutter speed (motion blur), or more realistic simulation of the optical components in a camera.

The algorithm works by describing illumination in a scene using the rendering equation, or light transport equation, and finding an approximate solution using Monte Carlo integration. An inefficient (but accurate) version of the algorithm can be very simple, and involves tracing a ray from the camera, allowing this ray to bounce in random directions as it hits different objects in the scene, and computing the amount of light transmitted along the path to the camera whenever the path encounters a light source. This process is repeated many times for each pixel (each repetition, with generated path and transmitted light, is called a sample), and the results are averaged. One main difference between this algorithm and standard ray tracing is that a single unbranching path is traced each time, while "Whitted-style" or "Cook-style" ray tracing recursively samples branching paths (e.g. when light is both reflected and refracted by a glass object).

More practical versions incorporate improvements such as quasi-Monte Carlo methods (techniques that distribute samples more evenly), importance sampling (take more samples of paths that are likely to transport more light), and next event estimation (allow a very limited form of branching, and sample additional paths that connect to the lights more directly).

Because path tracing uses random samples there is noise in the final image, which decreases as more samples are taken. Images commonly require many thousands of samples per pixel (spp) to reduce noise to an acceptable level, and denoising techniques (e.g. based on neural networks) are often used. Denoising is usually necessary when path tracing is used for real-time rendering in video games, because relatively few samples can be taken.

Many alternative algorithms for path tracing have been developed, although they do not always outperform more straightforward implementations. These include bidirectional path tracing (which traces paths forwards from the light source as well as backwards from the camera), Metropolis light transport, and ways of combining path tracing with photon mapping. Video games often use biased versions of path tracing to improve performance (e.g. limiting the number of bounces in each path). A family of techniques called ReSTIR has been developed that can help real-time path tracing by sharing data between nearby pixels and consecutive frames.

History

[edit]

The rendering equation and its use in computer graphics was presented by James Kajiya in 1986.[1] Path tracing was introduced then as an algorithm to find a numerical solution to the integral of the rendering equation. A decade later, Lafortune suggested many refinements, including bidirectional path tracing.[2]

Metropolis light transport, a method of perturbing previously found paths in order to increase performance for difficult scenes, was introduced in 1997 by Eric Veach and Leonidas J. Guibas.

More recently, CPUs and GPUs have become powerful enough to render images more quickly, causing more widespread interest in path tracing algorithms. Tim Purcell first presented a global illumination algorithm running on a GPU in 2002.[3] In February 2009, Austin Robison of Nvidia demonstrated the first commercial implementation of a path tracer running on a GPU,[4] and other implementations have followed, such as that of Vladimir Koylazov in August 2009.[5] This was aided by the maturing of GPGPU programming toolkits such as CUDA and OpenCL and GPU ray tracing SDKs such as OptiX.

Path tracing has played an important role in the film industry. Earlier films had relied on scanline rendering to produce CG visual effects and animation. In 1998, Blue Sky Studios rendered the Academy Award-winning short film Bunny with their proprietary CGI Studio path tracing renderer, featuring soft shadows and indirect illumination effects. Sony Pictures Imageworks' Monster House was, in 2006, the first animated feature film to be rendered entirely in a path tracer, using the commercial Arnold renderer. Also, Walt Disney Animation Studios has been using its own optimized path tracer known as Hyperion ever since the production of Big Hero 6 in 2014.[6] Pixar Animation Studios has also adopted path tracing for its commercial RenderMan renderer.

Description

[edit]
Noise decreases as the number of samples per pixel increase. The top left shows 1 sample per pixel, and doubles from left to right each square, until the last image at bottom right shows 32,768 samples per pixel.

Kajiya's 1986 paper defined a recursive integral equation called the rendering equation:[1]

This expresses I(x,x'), the light arriving at point x from point x', as the product of a geometry term, g(x,x'), which is 0 if there is something blocking the light between the two points and 1 otherwise, and the amount of light leaving point x' and traveling towards x. The light leaving point x' is the sum of the light emitted by the surface at x', and the integral of the light arriving at x' from all other points in the scene (the integration domain S) and being reflected towards x. The factor ρ(x,x',x''), which calculates how much light is reflected, must take into account the angles at which the light is arriving and leaving, and characteristics of the surface material, typically expressed as a bidirectional reflectance distribution function (BRDF) or bidirectional transmission distribution function (BTDF).[1][2]: 31.13 

The above equation assumes the light being evaluated is a single wavelength (it does not integrate over wavelengths, and wavelength does not appear as a parameter for any of the functions). It is a simplification for geometrical optics, and does not take into account phenomena such as diffraction, polarization, and fluorescence. It also (in the above form) only integrates over surfaces, and does not handle participating media (regions of space that scatter, absorb, and emit light). However extensions to handle such cases, and multiple wavelengths, were mentioned in the paper, and were well understood (there is no fundamental limitation of the path tracing algorithm that prevents rendering these effects).[1]

The equation is often called the light transport equation (LTE), and it is frequently written using integration over directions instead of surfaces. The geometry term ensures that only one surface point can contribute light from any given direction, even when integration over surfaces is used. Actual rendering also involves a related equation called the measurement equation that expresses the amount of light received by the camera along particular rays.[3][4][5][6]: 89 

In Whitted-style recursive ray tracing, the integral is replaced by a sum of the contributions of specific reflected and refracted rays, and the direct light arriving at the point from all light sources in the scene (lights may be point source or directional, so that lighting can be evaluated easily for any surface, or some approximation may be used for other types of lights). A constant "ambient term" may also be added to approximate global illumination. Distributed ray tracing introduced the use of Monte Carlo integration, rendering rough reflection and transmission by sampling multiple directions at each bounce, instead of using the contribution of a single direction, however diffuse global illumination was still usually approximated using an ambient term.[1]

Kajiya showed that his rendering equation is equivalent to the equation being solved by the radiosity rendering method, however the radiosity method was limited because it assumed all surfaces are Lambertian, and it required meshing of the surfaces. A method using Monte Carlo integration instead could be made unbiased for any scene.[1][6]: 9 

The problem with Monte Carlo integration is the high variance of the result (requiring many samples to reduce noise to an acceptable level). To address this, the paper first explored stratified sampling approaches. Distributed ray tracing allows multiple branching rays to be traced at each step, and a method called "hierarchical integration", using independent stratification of the integration domain at each step, had previously been explored. However global illumination requires sampling larger domains of directions or surface points. Adaptively splitting these domains in order to reduce variance was found to be too difficult. Importance sampling, where samples are chosen from a non-uniform distribution according to how much they are likely to contribute to the final result, was identified as a more viable approach.[1]

When using importance sampling, hierarchical integration (via recursive ray tracing) is no longer necessary, and this led to the formulation of path tracing:[1][2]: 31.17 

This diagram also points out an alternative algorithm for conventional distributed ray tracing. Rather than shooting a branching tree, just shoot a path with the rays chosen probabilistically. For scenes with much reflection and refraction, this cuts down vastly on the number of ray object intersections to be computed for a given pixel and performs a remarkable speed up of ray tracing for very little programming work. However, for this new fast form of ray tracing—called path tracing—we have found that it is very important to maintain the correct proportion of reflection, refraction, and shadow ray types contributing to each pixel.

— James T. Kajiya, "The Rendering Equation"[1]

Algorithm

[edit]

Ray tracers are often implemented using recursive functions, however a path tracer by nature does not need to be recursive, and can use a loop instead.

Here is pseudocode for a non-recursive function that traces a single path. It can either use next event estimation (NEE) or wait until the path hits a light by chance before accumulating light. It uses cosine-weighted sampling of ray directions, which is effective for diffuse surfaces (it is optimal for Lambertian surfaces) and simplifies the code:[7][8][9]

function TracePath(cameraRay):
    ray ← cameraRay
    throughput ← 1
    value ← 0

    loop:
        intersection ← FirstRayIntersection(ray)
        
        if [using NEE]:
            lightSample ← SampleLights()
            value ← value + throughput × ComputeLightAmount(ray, intersection, lightSample)
        else if [intersection object is a light]:
            value ← value + throughput × intersection.emittedLight
        
        newDirection ← CosineWeightedSample(intersection.surfaceNormal)

        throughput ← throughput × π × intersection.BRDF(−ray.direction, intersection.surfaceNormal, newDirection)
        
        ray.start ← intersection.location
        ray.direction ← newDirection
        
        q ← ComputeTerminationProbability(throughput)
        
        if random() < q:
            return value
        
        throughput ← throughput / (1 - q)

The code uses "ray" values that consist of a start point in 3D space and a normalized direction vector. The input camera ray should use the camera location as the start point, and a direction chosen based on the pixel location. The function should be called multiple times for each pixel, using the average of the returned results as the final pixel value. The random() function should return a value between 0 and 1.

For simplicity, the code only works for opaque surface materials, and it assumes the scene is enclosed (rays can never exit the scene). It only traces a single wavelength of light, but could be modified to work with RGB values or spectral rendering (see below). Light sources handled in the non-NEE case are assumed to be perfectly diffuse (otherwise, instead of a constant emittedLight value, another function would need to be evaluated to compute the amount of light emitted in the opposite direction to the ray).

The code uses a trick known as Russian roulette to make the function unbiased by terminating the loop probabilistically. Provided ComputeTerminationProbability returns a value greater than 0 (or eventually does, e.g. when throughput drops below a threshold) then the loop will eventually terminate (by returning the accumulated value). Whenever "q" is greater than 0 and the loop keeps going, throughput is increased to compensate for the fact that other sampled paths may have been terminated at that point. This ensures that the result has the correct expected value.[10]: 2.2.4 [11]: 13.2.1 [6]: 67-68 

The "q" value can be computed using any heuristic (even a constant probability could be used), but to make it effective, if throughput drops to a very low value (close to zero) then q should be close to 1. If larger termination probabilities are used, paths will tend to be shorter, but variance will be higher (there will be fewer long paths, but the throughput in those paths will be increased by a larger factor, which will tend to produce scattered bright pixels).[10]: 2.2.4 [12]: 5.3.2 

CosineWeightedSample samples directions in the hemisphere at a point on a surface, with probability proportional to the cosine of the angle between the direction and the surface normal (equivalent to integrating using the projected solid angle measure). This is commonly done for diffuse surfaces, and it simplifies the code because the cosine factor in the throughput formula and the inverse probability factor (required for importance sampling) cancel. Otherwise, direction sampling usually tries to use probabilities proportional to the product of the BRDF and the cosine factor. For perfectly specular surfaces, only a single direction is possible and slightly different code must be used.[6]: 77,82,172,254 [9]

The BRDF value is multiplied by π to cancel out the factor 1/π included in the BRDF by convention. (This convention comes from the fact that the integral of the cosine of the angle between the direction and the surface normal over the entire hemisphere is π, and so the BRDF for a perfect, non-absorbing, Lambertian surface must include a normalization factor of 1/π to make it evaluate to 1.) The 1/π factor can also be seen as part of the PDF for cosine-weighted sampling (which also includes the cosine factor that has been canceled), and since we need to divide by the PDF we multiply by π. Example path tracing code often explicitly computes the full PDF for cosine-weighted sampling and then divides by it (and multiplies the BRDF by the cosine factor, since it is no longer canceled) – this is also what the code needs to do if it uses a distribution other than the cosine-weighted distribution for sampling directions.[2]: 14.9.2 

Components of a modern path tracer

[edit]

To improve performance and turn path tracing into a useful algorithm for rendering complex, realistic images, various techniques and enhancements are incorporated into the rendering engine.

Sampling

[edit]

Importance sampling

[edit]

Importance sampling is a standard technique to reduce the variance (in this context, average squared magnitude of the error) of the result of Monte Carlo integration. It can significantly reduce the number of samples required to render a path traced image with an acceptable amount of noise. It involves choosing a probability density function (PDF) that is roughly proportional to the value being integrated, and drawing samples from that probability density instead of from a uniform distribution.[10]: 2.2.2 [6]: 47 

Surface materials do not reflect the same amount of light in all directions, and so a common type of importance sampling used in path tracing is to draw samples (in this case, directions from a hemisphere around a surface point) from a PDF that has the same shape as the bidirectional reflectance distribution function (BRDF) used for the material (see #Scattering distribution functions below) when choosing a direction for the next ray in a path. This is especially effective for glossy or shiny materials.[6]: 254 

Multiple importance sampling

[edit]

Importance sampling requires selecting a single probability density function (PDF) from which to draw the samples used for Monte Carlo integration, but often in path tracing, there are multiple alternative PDFs that can be used. For example, this can occur when integrating a function that is the product of two other functions, if there are good PDFs known for sampling the two functions individually, but not for sampling the product. Either one of the PDFs must be chosen, or some method must be found for combining them.[10]: 2.2.3 

A common case is sampling the amount of direct light from an area light that is reflected by a point on the surface of an object towards the camera. When choosing the ray direction to sample at the object's surface point, either a random point on the surface of the light can be chosen (without using information about the object's surface material) and then the direction of the ray towards that light point can be computed, or a ray direction can be sampled proportional to the bidirectional reflectance distribution function (BRDF) of the object's surface (without taking the light location and shape into consideration). The first strategy ensures that the sampled ray does not miss the light, but for a glossy surface it may choose a direction at which very little light is reflected towards the camera, so it is more effective for rough (diffuse) surfaces. The second strategy is often effective for glossy surfaces because it will likely choose a direction at which the value of the BRDF is high, but the ray in the chosen direction may miss the light.[6]: 252-257 

In simpler forms of path tracing, usually the first strategy is used for direct light, and is known as next event estimation (NEE), and the algorithm then continues tracing the path to sample indirect light, using the second strategy to select the sampled direction. If the next ray hits the surface of the light by chance (the light is usually also an object in the scene), the emitted light from the surface is ignored because otherwise that light would be counted twice. This works well for diffuse surfaces, but is not optimal for glossy surfaces.[11]: 13.2.3 

A method called multiple importance sampling (MIS) allows using both types of sampling together, avoiding situations where a suboptimal strategy is chosen or values are counted twice (causing bias). MIS computes weights for blending the two samples so that the result is unbiased, and a larger weight is given to the strategy that is more likely to be effective. Weights are chosen using a heuristic, and an unbiased result will be produced if the weights satisfy certain conditions. Although various heuristics have been devised, some of which may be more effective in different situations, the balance heuristic is broadly applicable. A variation called the one-sample model selects one sampling strategy to use instead of blending multiple samples, and in this variation the balance heuristic always minimizes variance.[6]: 252-287 [10]: 2.2.3 

MIS can be used whenever multiple alternative probability density functions (PDFs) are available to use for importance sampling, and none of the alternatives is clearly better in all situations (some of the PDFs might only partially cover the domain being sampled). It is an essential component of bidirectional path tracing (see below).[6]: 251,297 

Quasi-Monte Carlo sampling

[edit]

The basic path tracing algorithm uses a random number generator to choose ray directions to sample at each point in the path (and points on lights, if next event estimation is used). This ensures that the result is unbiased, and avoids creating visible artifacts in the image. However when path tracing is repeated and the samples are averaged, because the sampled directions are completely random, they will not sample the integration domain evenly (there will be clumps and gaps in the directions that are sampled). This is normal and unavoidable for standard Monte Carlo integration, but there are techniques called quasi-Monte Carlo techniques that can reduce the variance by making the samples more evenly distributed. This means that the samples no longer have the randomness properties required by standard Monte Carlo integration, and so it must be done carefully to avoid bias and artifacts.[13]: 8.2.2 

The simplest way of doing this is to break the domain being sampled (e.g. the hemisphere of ray directions leaving a surface) into segments and choosing a random point in each segment. This is called stratified sampling, and its drawbacks are that the number of samples being taken must be known in advance, and clumping may still occur (samples from adjacent segments may be very close together if they are both at the edge of a segment).[10]: 2.2.1 [14]

If the number of samples is not known in advance, methods called low-discrepancy sequences can be used to generate samples that tend to be spread out and cover the domain evenly. Popular sequences with this property include Halton and Sobol' sequences. These are predefined (non-random) sequences, but in modern path tracers the sequences are permuted (scrambled) to make them more random.[15][16]

Stratified sampling and low-discrepancy sequences can become less effective as the dimension of the domain being sampled increases. Path tracing involves generating very high-dimensional samples. E.g. simple path tracing of opaque surfaces (with no light sampling) uses samples that have dimension equal to twice the number of bounces in the path, because each ray direction is a 2D sample on a hemisphere. If stratified sampling or a low-discrepancy sequence is used independently for each bounce, there will likely be correlations between the bounce directions, which will cause bias in the image, and also the complete samples (including all bounce directions) will not cover the domain evenly. Path tracers typically use very high dimensional low discrepancy sequences, but this means that sampling of individual bounces (considered independently as 2D samples) may not be optimal. A simple technique called padding improves this, and a more expensive technique called sliced optimal transport generates sequences where all lower-dimensional projections of the multidimensional samples tend to be evenly distributed, by repeatedly optimizing 1D slices.[14][17]

The above techniques ensure that the set of paths sampled for each individual pixel is well-distributed. Techniques based on blue noise can help ensure that samples for adjacent pixels tend to be widely separated. This is usually more important when only a small number of paths is traced per pixel.[18]: 8.1.6 [16]: 8.7.6 

Spectral rendering

[edit]

Computer graphics commonly represents light values as vectors of red, green, and blue components. Color or albedo for surfaces is similarly represented as separate multipliers for red, green, and blue. Although this approach can be adequate in some situations (e.g. it is accurate when light values are added together), when applied to path tracing it is equivalent to simulating a scene lit by exactly three wavelengths of light, which is not realistic. Modern path tracers provide the option to instead use random samples of wavelengths in the visible spectrum of light, which is called spectral rendering. This is now a standard approach when realistic, physically-based rendering is the goal.[2]: 331 [19]

For performance reasons, multiple (e.g. four) wavelengths are usually sampled at a time (a single path is traced, but the lighting calculations are performed for multiple wavelengths). This can also reduce the color noise that occurs with spectral sampling. Wavelengths are typically converted into RGB values when the path contributions are accumulated.[19]

This approach allows simulating chromatic dispersion (e.g. "rainbows" visible when light hits a cut glass object). It also allows rendering scenes with more realistic light sources and materials, and correctly modeling diffuse global illumination and participating media, when light is bounced multiple times between colored surfaces or scattered by colored volumetric media (without exaggerated color or unnatural color shifts).[19]

Spatial data structures

[edit]

Ray-object intersection acceleration

[edit]

One of the most common and expensive operations performed by a path tracer is finding the first object, and corresponding surface point, intersected by a light ray starting at a particular point and traveling in a particular direction (a similar but simpler operation is testing if there is anything blocking the light between two points in space). To speed this up, path tracers use a 3D spatial data structure. The structure most commonly used today is the bounding volume hierarchy (BVH), and recent GPUs provide a hardware-accelerated version of this. However other structures such as k-d trees and octrees have also been used. Some path tracers require that all objects be represented using triangle meshes, which simplifies code and data structures (hardware acceleration is also usually optimized for triangles).[20][2]: 1067-1107 [21]: xvii 

Light sampling acceleration

[edit]

Generating samples of direct light (for next event estimation) is most effective if importance sampling is used, which requires sampling lights proportionally to their brightness, and (ideally) inversely proportionally to the square of the distance to the light. When the scene has a large number of lights, it becomes prohibitively expensive to examine all lights and compute weights before choosing a small number of lights to sample.[22]

BVH light sampling, using a spatial tree data structure, can speed up sampling while taking locations of lights into account (an approach that only considers brightness, called power sampling, uses a simpler non-spatial data structure). Lights that are close together are grouped together in the tree, and the total brightness of the lights in each branch of the tree is stored. Spatial bounds for the branches of the tree allow quickly skipping lights that are behind the surface for which direct light is being estimated, and also allow weighting groups of lights based on direction. Use of a tree for sampling is unbiased because the aggregate data is only used to compute sampling probabilities, not for the light amount used for rendering. A biased algorithm called lightcuts uses a similar tree, but approximates groups of distant lights (branches of the tree) by treating them as a single light, instead of importance sampling individual lights.[22][12]: 7.8 

Turning samples into images

[edit]

Filtering

[edit]

If path tracing only uses camera rays corresponding to the exact centers of pixels in the image, severe aliasing can result. Path tracers usually remove or reduce aliasing by filtering out frequencies higher than can be represented by the image's resolution. To do this, they use a filter kernel, such as a Gaussian filter, or filters such as Mitchell–Netravali filters or the Lanczos filter that improve sharpness but can introduce ringing artifacts. Either locations around the pixel center are sampled, weighted by the filter kernel (called filter importance sampling), or locations on the camera plane are sampled uniformly (usually with stratified sampling, so the same number of samples is taken in the square around each pixel) and then samples are added to any pixels whose kernels intersect the sample (called weighted importance sampling). The latter approach can reduce variance slightly, but may not be compatible with denoisers (for example, the occasional very bright samples called "fireflies" are spread across a neighborhood of pixels, and cannot be easily removed).[5]: 5.4.3 [23]

Denoising

[edit]

Even when thousands of paths are averaged, the output of path tracing usually contains visible noise. Although the most reliable (and unbiased) way to reduce noise is to trace more paths (or make improvements such as better importance sampling), many biased approaches have been used to eliminate or reduce noise.[24][25]

Path traced images often contain bright pixels colloquially called "fireflies", which can result when a path that was considered low priority by importance sampling (unlikely to produce a bright value) intersects a light source or a bright area of the scene, or when indirect light enters through a narrow gap and very few paths travel through that gap. Although technically these bright samples are necessary in order to produce an unbiased image, they are subjectively unhelpful, and they can greatly increase variance. Many heuristics have been used to identify fireflies and remove them, e.g. replacing them with the average of surrounding pixels or spreading their brightness out to nearby pixels. This is similar to outlier detection in statistics. Sometimes individual samples are filtered, rather than accumulated pixel values. Techniques related to density estimation, using variable-width kernels, have been tried.[24]

Many techniques use edge detection to allow denoising filters that blur the image while still preserving sharp edges of objects. Edge detection for denoising a rendered image typically uses data other than the final brightness, such as the depth and normal vector of the first point intersected by the camera ray. Edge-preserving filters that do not rely on this type of data (using only pixel color and brightness values) include bilateral filters, guided filters, and non-local means filters. Techniques based on wavelets and other multiscale filters have been tried, which can handle the varying frequency characteristics of an image (e.g. blurriness caused by depth of field).[24][25]

Sometimes the direct light and indirect light contributions to a pixel are rendered and denoised separately, which allows preserving detail in the direct lighting, while smoothing the indirect light (which is usually noisier). More complex algorithms denoise specular reflection separately (since it varies differently with pixel location), and some work with individual samples and construct a statistical model for the local light field around each visible point on a surface.[24][25]

More recently, convolutional neural networks have been used to implement denoising filters, training the neural network on a large dataset of noisy and noiseless copies of the same images. These neural networks can use data such as depth and normal, surface albedo, and surface roughness/shininess as part of their inputs.[24]

An alternative or complementary approach is adaptive sampling, which takes additional samples in regions of the image where variance is higher (this is tricky to do without introducing bias).[26][25]

Tone mapping

[edit]

The output of path tracing is linear brightness values for pixels, which then need to be converted into a viewable image or a video or film frame. The range of brightness values may be larger than displayable by a standard computer monitor, or representable by a file format that uses 8-bit color values, such as JPEG or PNG. Images often contain very bright highlights, or pixels where a light source such as the sun is seen directly. If the brightness of the entire image is reduced so that the brightest pixels are not clipped, the rest of the scene will often be too dark.[27]: Ch20 

Methods called tone mapping or tone reproduction are often used to transform (or map) the pixel values in such high dynamic range (HDR) images into the range of values allowed by common file formats and computer monitors or other displays, while attempting to make details in both shadows and brighter areas visible, often incorporating models of how human vision responds to varying brightness. The aim is usually to produce a subjective impression of the true brightness of the scene, despite the reduced range of pixel values. Many different methods and tone mapping curves are available, often with parameters to tune them. Some methods aim to reproduce the look of images captured on photographic film, and may alter the colors and saturation of the image.[28]: 8.2.2 [27]: Ch20 

Tone mapping does not need to be implemented as part of the path tracer. If the rendered image is output to a linear HDR format (e.g. OpenEXR), tone mapping can be applied later (which may be preferable, since the tone mapping method or parameters can be changed without re-rendering the image).

Scattering distribution functions

[edit]

Real objects are never perfectly smooth mirrors (although perfect mirrors are often simulated in computer graphics for convenience and simplicity). When light hits an opaque object, it is always reflected (or scattered) in multiple directions. The amount of light that is reflected in each direction mainly depends on the angle at which it arrives relative to the surface normal. Shiny materials, often called specular materials, reflect more light at angles close to the angle at which a perfect mirror would reflect light, producing a distinct highlight (called a specular highlight) when a bright light source is reflected. In contrast, materials that scatter light more (reflecting it in a broader range of directions without a sharp peak) are called rough or diffuse. In general there is no clear distinction between specular and diffuse materials, for example a diffuse surface may have a coat of varnish over it that gives it specular highlights.[29][30][2]: 26.10,Ch27 

A photograph of brushed stainless steel appliances. The anisotropy of the material changes the shape of highlights and reflections, stretching or blurring them along a particular direction.

Materials for which the amount of light reflected in a particular direction depends only on the angles between the incoming ray of light (called the incident ray), the outgoing or reflected ray (called exitant), and the surface normal are called isotropic. In this case the surface material has no particular orientation (rotating the surface around the surface normal will not change the way light is reflected). Materials for which the orientation does matter are called anisotropic, and include materials such as brushed steel or aluminum. Although many real-world materials are subtly anisotropic, anisotropy is often ignored by computer graphics, since it is simpler and more efficient to treat materials as isotropic.[29][30]

One extreme case is a "perfectly rough" material where the amount of reflected light does not depend on the angle of the incident light, and there are no highlights at all. An example of this type of material is the Lambertian material commonly used in computer graphics (it does not exist in the real world, but a material called Spectralon, which is a polymer powder, is very close). For a Lambertian material, the amount of reflected light is proportional to the cosine of the angle between the exitant ray and the surface normal.[9][2]: 27.5.2 

The above effects can all be described by a type of function called a bidirectional reflectance distribution function (BRDF). The parameters to these functions are the angles of the incoming ray and the outgoing ray with respect to some frame of reference on the surface. The output of the function is a factor that the amount of incoming light should be multiplied by to get the amount of outgoing light. The BRDF, as typically defined, does not include the factor for Lambertian refectance mentioned above (the cosine of the angle between the exitant ray and the surface normal) and so the BRDF for a Lambertian surface is a constant function. Omitting this factor gives the BRDF the property of reciprocity: swapping the incident and exitant rays in the parameters to the function does not change the output (by convention, both directions point away from the surface). The BRDF is scaled appropriately for use in the light transport equations (where it is multiplied by incoming light values and integrated), and it returns a value of 1/π for a Lambertian material that does not absorb any light.[29][2]: 26.10,Ch27 [6]: 3.6 

Along with reciprocity, an important property of valid BRDFs is that they are energy conserving: when the BRDF is used in light transport equations, the total amount of light reflected should never be greater than the amount of light arriving at the surface. The BRDF must also always return a non-negative value. (These properties will not hold for individual wavelengths when effects such as fluorescence are observed.)[29][2]: Ch27 

BRDFs can be different for each wavelength of light. For example non-metallic materials usually have white highlights, reflecting all wavelengths nearly equally at certain angles, while also diffusely reflecting light in a wavelength-dependent way (which gives the material its color). Wavelength-varying BRDFs can also model iridescent materials.[29][31]

BRDFs are a core part of path tracing: they are evaluated at every path vertex where a ray intersects a surface (in order to compute the throughput of the remaining part of the path), and also when computing direct lighting at surface points. BRDFs do not describe properties of transparent or partially transparent materials (for which other types of scattering distribution functions are needed). However BRDFs are useful because many materials are effectively opaque to visible light, or have a very small amount of translucency near the surface that can be ignored or approximated by the BRDF.[29]

Although data for BRDFs can be obtained by measuring reflectance of real surfaces, the BRDFs used for rendering in practice are usually mathematical functions (often designed to simulate simplified physical properties of real surfaces) that are reasonably efficient to evaluate, and produce effects similar to real materials. They often have parameters such as "roughness" that can be easily changed. Multiple simple BRDFs (often called lobes due to the shape of the functions) are sometimes layered.[30][2]: Ch27 

BRDFs do not, by themselves, include information about the texture of a surface. In realistic rendering, often color and "roughness" parameters (and sometimes orientation and amount of anisotropy) are encoded using texture mapping and used to modify the BRDF for individual points on the surface.[30][32][2]: 27.15 

A similar type of function used when light passes through the surface of a transparent material is a bidirectional transmittance distribution function (BTDF). A term that includes both BRDFs and BTDFs is bidirectional scattering distribution function (BSDF). To handle general scattering in transparent materials, volume rendering uses a similar function (but with an incompatible definition) called a phase function.[29][33][2]: 14.9 

Volume rendering

[edit]

Volumetric path tracing is used for rendering scenes containing participating media, which are 3D regions of space (rather than the surfaces of objects) that emit light, or in which light can be absorbed or scattered. Participating media are often thought of as containing particles, so that a ray of light passing through the medium has a chance of interacting with a particle (and being absorbed or scattered) that is higher the farther the ray travels. Path tracers that perform volume rendering usually allow it to be combined with rendering of surfaces. Kayija's rendering equation can be generalized for volumetric path tracing, however there are other volumetric light transport equations that may lead to more practical algorithms (just as the original form of the rendering equation is usually converted to use integrals over ray directions for standard path tracing).[34][33][1]

Volumetric path tracing is commonly performed backwards from the camera, like standard path tracing, taking advantage of the reciprocity property of most light interactions. Camera rays are sampled (or ray directions for reflections or global illumination that arrives at a point on a surface) as usual, and then scattering events are generated when a ray passes through a participating medium that can scatter light. The probability of these scattering events (and how far the ray usually travels before the next event) is related to the density of the material. When a ray is scattered, a new ray is cast in the new direction, similar to scattering from a surface, with the probability of each new direction determined by a probability density function called a phase function.[34][33]

Volumetric path tracing also uses integration to compute how much light is absorbed and scattered away when a ray passes through space (called attenuation), and how much light is emitted back along the ray (taking absorption and scattering along the ray into account) if the participating medium is emissive.[33]

All of these operations are much simpler and faster if the participating medium is homogeneous (has the same density everywhere with respect to scattering, absorption, and emission). In that case integration can be performed analytically to compute attenuation and emission, and there is also a simpler way to generate scattering events with the correct probability distribution. Materials typically rendered as homogeneous media (which may still have complex shapes) include water and colored glass. The formula used for homogeneous absorption is known as the Beer–Lambert law or Beer's law.[33]

When the participating medium is inhomogenous (has density or other properties that vary spatially) more complex and slower algorithms must be used. These types of media include clouds, mist, smoke, and fire. The input data for rendering these materials is usually volumetric data (structured similarly to voxels). To perform operations such as integration, the code often needs to ray march through the media, querying data for points along the ray. This can be done sparsely (examining the material at only a few, usually randomly chosen, points) or densely (examining closely spaced points), depending on the algorithm. The commonly used null scattering algorithm treats the medium as homogeneous when choosing which points to sample, allowing it to sample the medium sparsely, and it can be very efficient if the medium is nearly homogeneous.[35][36]

Bidirectional path tracing

[edit]

Sampling the integral can be done by either of the following two distinct approaches:

  • Backwards path tracing, where paths are generated starting from the camera and bouncing around the scene until they encounter a light source. This is referred to as "backwards" because starting paths from the camera and moving towards the light source is opposite the direction that the light is actually traveling. It still produces the same result because all optical systems are reversible.
  • Light tracing (or forwards path tracing), where paths are generated starting from the light sources and bouncing around the scene until they encounter the camera.

In both cases, a technique called next event estimation can be used to reduce variance. This works by directly sampling an important feature (the camera in the case of light tracing, or a light source in the case of backwards path tracing) instead of waiting for a path to hit it by chance. This technique is usually effective, but becomes less useful when specular or near-specular BRDFs are present. For backwards path tracing, this creates high variance for caustic paths that interact with a diffuse surface, then bounce off a specular surface before hitting a light source. Next event estimation cannot be used to sample these paths directly from the diffuse surface, because the specular interaction is in the middle. Likewise, it cannot be used to sample paths from the specular surface because there is only one direction that the light can bounce. Light tracing has a similar issue when paths interact with a specular surface before hitting the camera. Because this situation is significantly more common, and noisy (or completely black) glass objects are very visually disruptive, backwards path tracing is the only method that is used for unidirectional path tracing in practice.

Bidirectional path tracing provides an algorithm that combines the two approaches and can produce lower variance than either method alone. For each sample, two paths are traced independently: one from the light source and one from the camera. This produces a set of possible sampling strategies, where every vertex of one path can be connected directly to every vertex of the other. The original light tracing and backwards path tracing algorithms are both special cases of these strategies. For light tracing, it is connecting the vertices of the camera path directly to the first vertex of the light path. For backwards path tracing, it is connecting the vertices of the light path to the first vertex of the camera path. In addition, there are several completely new sampling strategies, where intermediate vertices are connected. Weighting all of these sampling strategies using multiple importance sampling creates a new sampler that can converge faster than unidirectional path tracing, even though more work is required for each sample. This works particularly well for caustics or scenes that are lit primarily through indirect lighting.

Metropolis light transport

[edit]

Importance sampling can be used to improve the efficiency of Monte Carlo integration by using a sample distribution that is roughly proportional to the value being integrated. However this only works if a good distribution is known, and there is a practical way to draw samples from it. The sample distributions used in path tracing are often far from optimal, especially if rays starting from the camera must reflect or refract in order to reach the light. An alternative way to perform MC integration, the Metropolis method, uses a random walk to generate samples with an improved distribution. It is a Markov chain Monte Carlo (MCMC) method dating back to 1953 (extended by Hastings in 1970), but its first published use in path tracing, called Metropolis light transport (MLT), was not until 1997, by Veach and Guibas (although other researchers had previously explored using MCMC to find the equilibrium state for light transport).[37][38][1]

In standard importance sampling, samples are generated independently, e.g. by sampling ray directions and points on area lights inside a path tracing function that takes a list of random numbers as input (or uses successive calls to a pseudorandom number generator). With the Metropolis method, samples (called proposals) are instead generated from other samples by modifying them randomly. In the context of MLT, the samples are complete paths from the camera to the light source, and the proposals are called mutations (of paths). The main benefit of the algorithm comes from being able to generate new paths that are "close" to existing paths, so they are likely to be good paths for importance sampling if the original path was good. If these paths were used in standard importance sampling, the path tracer would need to know the absolute probability of generating each path, which is likely intractable to compute, but the Metropolis method only requires knowing the conditional probability of generating each mutated path given the previous path (and the conditional probability of the reverse transformation).[37][38][39]

A target function (typically the luminance of the light contributed to the image by a given path) is used to guide sampling. After a new path has been generated from an existing path, the target function is evaluated for the new path and then the Metropolis method provides a formula for deciding (probabilistically) whether to use the new path (continuing the random walk in "path space" in that direction) or stick with the previous path and try again. Paths (proposals) that are chosen are said to be accepted. Paths for which the target function is higher are more likely to be accepted, and paths that are invalid (e.g. some edge in the path is blocked by another object) are never accepted.[37][38][39]

While the Metropolis method could in theory be applied to standard unidirectional path tracing (starting from the camera) and used to generate samples for a single pixel at a time, Veach and Guibas were interested in using it for "difficult" lighting scenarios where bidirectional path tracing is typically helpful; a required initial path generation step and normalization step (estimating the overall brightness of the image) would also make the method inefficient for single pixels. MLT allows generated paths to "wander" all over the image, contributing to any pixel (similar to the paths used for caustics in bidirectional path tracing).[38][6]

MLT needs to generate an initial unbiased sample to use as a "seed" path before it can start generating mutations. This is usually done by resampling, generating (weighted) samples with a distribution proportional to the target function. New seed paths are used every certain number of mutations to help ensure that the entire path space is sampled adequately. The original algorithm selected seed paths from sets of paths generated by bidirectional path tracing.[38][6]

Mutations used in the original MLT algorithm[38][6]
In the following descriptions, a "specular vertex" is a surface point where reflection or refraction occurs with very little scattering of light (e.g. a point on the surface of glass or water, or on a mirror), and a "diffuse vertex" is a point in a path that scatters light diffusely towards the camera.

  • bidirectional mutation – a randomly chosen portion of the path (at either end, or in the middle, perhaps of zero length) is replaced by a new subpath (perhaps of zero length).
  • lens perturbation – the first non-specular vertex after the camera is shifted slightly by changing the direction of the camera ray and finding the new first vertex on a diffuse surface.
  • caustic perturbation – similar to lens perturbation but skips one diffuse vertex immediately after the camera, and any following specular vertices, and then perturbs the direction of the light ray coming back from either the light source or the next diffuse vertex.
  • multi-chain perturbation – allows perturbing a diffuse vertex that is reached via specular vertices from both the camera and the light source (e.g. for caustics on the bottom of a pool)
  • lens subpath mutation – replace the camera ray and any following specular vertices in order to produce a path for a new pixel (usually a nearby pixel). This replacement is done in a stratified way to ensure coverage of the entire image.

Primary sample space Metropolis light transport

[edit]

Because MLT, as originally proposed, generates paths in a very different way than standard path tracing (including bidirectional path tracing) it can be difficult to add MLT to existing path tracers, or incorporate it as part of a path tracer that performs multiple types of path tracing. Primary sample space Metropolis light transport (PSSMLT) solves this by taking advantage of the fact that in standard path tracing sample generation is driven by a supply of random numbers (or quasi-random numbers from low-discrepancy sequences) which are used to select ray directions and other random variables at each stage of path tracing. The sequence of random numbers, usually numbers in the interval [0, 1), used to generate a particular individual sample is a representation of the sample in a domain called primary sample space. Path tracing can be viewed as Monte Carlo integration using uniform sampling in this sample space (instead of using importance sampling in path space).[37][39]

Since path tracers typically already provide multiple ways to generate these random numbers, it is usually possible to allow generation of the numbers to be driven by Metropolis method. This allows an implementation of MLT to use standard path tracing code to evaluate the samples, greatly reducing the amount of new code that needs to be written. Mutated paths can be generated by modifying the random numbers slightly, rather than performing complex operations on paths. Because the transition probabilities from path to path in PSSMLT are usually symmetrical (probability is the same in both directions), the calculations are much simpler.[37][39]

However PSSMLT is often much less effective than traditional MTL because it cannot usually mutate a small part of the path and keep the rest (which greatly improves efficiency for exploring long paths in MLT). Instead of the five specialized mutation types used by traditional MLT (e.g. adding or removing one or more path vertices in the middle of a path) it usually just uses large and small steps that modify all of the numbers.[37][39]

Multiplexed Metropolis light transport (MMLT), which is designed for primary sample space (and thus is still much easier to implement than traditional MLT but cannot generate all types of mutations) applies the multiple importance sampling (MIS) balance heuristic to choose between multiple bidirectional path tracing strategies for connecting paths. This functions as an extra dimension in the sample space. MMLT still does not always outperform traditional MLT, but can be more robust, reducing correlation artifacts.[37][39]

Photon mapping

[edit]
3D rendered image showing three copies of a cartoon cow. The one on the left has a mirror surface, and the one on the right uses a transparent glass material. The base surface is illuminated by finely detailed bright spots and lines ("caustics") caused by light being focused by the reflective and transparent cows. The caustics are colorful in some places, due to chromatic dispersion.
An image rendered in POV-Ray using photon mapping (not using path tracing) showing detailed caustics. Faint patterns are visible in the caustics due to the light sampling pattern used by POV-Ray (which reduces noise but causes artifacts).

Path tracing has difficulty rendering caustics – bright patches of light that appear when a small or highly directional light source (such as the sun) is reflected or refracted by a shiny surface or transparent object before illuminating another surface. The result is usually very noisy, with many "fireflies" (stray bright pixels), or caustics may simply be missing. Bidirectional path tracing can render caustics if light subpaths are allowed to contribute to any pixel in the image (breaking the pixel-by-pixel rendering model), but caustics may still be missing if viewed in a mirror. This type of difficult lighting can also cause noise elsewhere in the image.[40][6][41]: 16.2.2 

A different rendering technique called photon mapping or photon density estimation is able to render caustics more effectively. Photon mapping works like the "forward" or "particle tracing" portion of bidirectional path tracing, simulating the paths of many "packets" of light emitted by a light source. These are called photons in the algorithm, but they usually do not actually represent physical photons. Photons are traced through reflections and refractions until they hit a diffuse (matte) surface or leave the scene. Information about photons that hit a surface is accumulated in a data structure called a photon map and used later, rather than being used immediately for rendering a particular pixel. The photon map is implemented using a spatial structure such as a k-d tree or hash grid. Photon mapping is biased, because it uses a density estimation filter to smooth out the data and allow estimating the intensity of light at an arbitrary point on a surface. If an insufficient number of light samples is used, the image may have a blurry or blotchy appearance caused by the filter. Photon mapping is usually combined with some form of ray tracing, allowing precise direct illumination and reflection and refraction of camera rays, while photon maps are used for only for caustics and/or diffuse global illumination (often with separate maps for each, because caustics require denser sampling).[12]: 7.6 [2]: 31.19,32.6 [40]

One early variation of photon mapping used a photon map to "guide" rays for global illumination (which was rendered using distribution ray tracing), instead of using the radiance information in the map directly. This is similar to path guiding in modern path tracing. Another method explored was the use of data from photon maps as control variates for reducing the variance of path tracing. Both of these approaches can be unbiased.[40][42]: 2 

Progressive photon mapping

[edit]

An algorithm called progressive photon mapping is optimized to allow tracing a larger number of photons, with the number not limited by available memory. Tracing more photons can reduce bias. Rather than beginning with photon mapping, it starts with a "camera pass", in which paths are traced from the camera until they hit a diffuse surface. Information about these subpaths (called visible points) is stored in a spatial data structure. In the photon mapping pass, subpaths from the light source are traced until they hit a diffuse surface, and any nearby subpaths from the camera pass are then found. The camera subpaths determine how much light from the light subpath is added to each pixel in the image.[41]

Instead of consuming memory by storing photons (light subpaths) in a k-d tree (or other spatial structure), progressive photon mapping stores visible points (camera subpaths), which is still a problem because a path traced image usually requires a large number of camera rays for each pixel. A variation called stochastic progressive photon mapping makes the algorithm more practical by breaking rendering into multiple pairs of camera and light passes, storing typically only a single camera subpath for each pixel at a time. The algorithm also tracks the number of nearby photons found for each pixel's visible points, and reduces the size of the filter kernel in subsequent passes, which allows rendering sharp caustics even though the photons are traced in smaller batches.[41]

Unified frameworks

[edit]

One problem with photon mapping approaches is that surfaces must usually be divided into specular (glossy) and diffuse surfaces, with different strategies used for each, and this classification may not always be optimal. Later techniques called vertex connection and merging (VCM) and unified path sampling (UPS) create frameworks that allow using both photon mapping and standard path tracing on any surface, and attempt to reduce bias further.[43][44][45]

The unified frameworks are forms of bidirectional path tracing (BDPT), based on combining paths generated by different strategies. The same path can potentially be generated by multiple strategies, and so paths must be blended carefully, using multiple importance sampling (MIS) weights. Photon mapping samples are represented as paths where a subpath from the camera and a subpath from a light source have been joined at nearby (but not exactly matching) points on a surface, or as paths where a vertex on a surface has been perturbed using a probability distribution corresponding to the filter kernel that was used to smooth photon density estimation. Because the sampling probabilities of all types of paths can be computed (whether or not they use the photon mapping strategy), they can be blended using MIS weights, as in standard BDPT. The algorithm is still biased but will converge to the correct result faster than other forms of photon mapping as the number of light subpaths increases.[43][44]

Reusing the same light subpath for multiple pixels (as is done in most forms of photon mapping) can cause correlations between nearby pixels, which may produce visible artifacts and cause problems for denoisers. The VCM method reduces this correlation by randomly modifying each light subpath before using it.[43]

ReSTIR

[edit]

Improvements to GPU performance and ray tracing acceleration have enabled the widespread use of ray tracing in video games, but in order to render realistic lighting in scenes lit by large numbers of lights, and diffuse global illumination with many bounces, many rays must be cast. This is challenging to do in real time at an acceptable frame rate, especially when rendering large, detailed environments (in particular, unbiased path tracing, with arbitrary path lengths, is usually not practical in real time). One approach to address this is to share data between pixels in the image, or between consecutive frames, so fewer ray casts need to be performed per pixel. This is commonly done by denoisers and temporal anti-aliasing (which both enable using fewer samples per pixel), however they can cause blurring of the image, "ghosting" artifacts across frames, light leakage around corners, and other types of bias. Reuse algorithms based on precomputation and caching may not respond quickly enough to lighting changes during real-time rendering, and may cause significant bias.[46]

A family of techniques called spatiotemporal reservoir resampling, or ReSTIR, allows sharing of samples between nearby pixels, and carrying samples forward to the next frame even when objects or the camera have moved. It can do this without introducing bias, or it can be used in biased modes that have better performance and less noise. ReSTIR consists of steps to generate new direct light or global illumination samples for a particular pixel, spatial reuse steps that try to reuse samples from nearby pixels, and temporal reuse steps that try to reuse samples from the previous frame (using optical flow, also known as temporal reprojection or backprojection, to choose a corresponding pixel, accounting for movement of the camera and scene objects).[46][47]: 2,10 

ReSTIR uses a form of weighted reservoir sampling, maintaining a data structure called a reservoir for each pixel. The reservoir stores a single sample along with a weight similar to the weights used in importance sampling. (Reservoirs storing more than one sample can also be used: the original ReSTIR implementation used 4 samples per reservoir for its biased version but only 1 sample for the more expensive unbiased version).[48]: 15 [47]: 10 

The earliest form of ReSTIR, now called ReSTIR DI, is designed for direct lighting in scenes with many lights. To generate initial samples, it uses an older technique called resampled importance sampling (RIS). It take many samples of lights cheaply (without considering visibility) and chooses one based on how bright the samples are (this is an approximate or stochastic form of importance sampling). The chosen sample (e.g. a point on a specific light) is then tested to determine if it is visible (if not, it can be treated as a light sample with zero brightness) and is stored in the reservoir, along with a weight.[47]: 4-6 [48]: 18 

During spatial reuse steps, another pixel is chosen randomly from a neighborhood around the target pixel (e.g. all pixels with distance less than 30 pixels away). The sample in this other pixel's reservoir is then merged into the target pixel's reservoir by computing multiple importance sampling (MIS) weights for the two samples and choosing a sample using probabilities proportional to the weights (a similar procedure to RIS). The weight in the reservoir is updated whether or not the other pixel's sample was chosen. This process is expensive in unbiased versions, because computing the MIS weights requires knowing visibility and brightness for both of the light samples, at the surface points corresponding to both pixels, which usually involves tracing two rays or paths (this cost can be reduced by using a reciprocal reuse pattern). Some implementations examine multiple neighboring pixels (e.g. 4 neighbors) and choose one, which helps reduce variance but is more expensive, requiring tracing more rays.[47]: 5-6 [48]: 12,16 

Temporal reuse, reusing samples from a corresponding pixel in the previous frame, is implemented in a similar way to spatial reuse. Because the camera and objects in the scene may have moved and lighting may have changed, visibility and brightness of both light samples at the surface point in both frames needs to be computed in order to calculate the correct weights, again usually requiring tracing two additional rays.[47]: 6 [48]: 16,19 

A typical sequence of steps per frame (likely requiring multiple passes over all pixels) would be:[48]: 16-17 

  1. Generate new initial samples for pixels
  2. Perform temporal reuse
  3. Perform spatial reuse (one or more times)
  4. Render the pixels for the current frame using the samples now in the reservoirs
  5. Perform denoising of the rendered image

By combining the steps in this way, samples can quickly propagate to neighboring pixels allowing a large amount of potential reuse of any "good" samples that are found. This can be helpful in difficult lighting situations (e.g. when there are many lights but most of them are shadowed) however excessive reuse of a small number of samples causes correlation artifacts that can be very visible.[48]: 16-17 [49]: 16-17 

Path tracing can use the unbiased version of ReSTIR DI to compute direct light for the first surface point intersected by the camera ray. A more general version of ReSTIR for path tracing, called ReSTIR PT, extends the algorithm to support reusing arbitrary paths instead of just direct light samples. This involves applying a shift mapping transformation to a sample to make it applicable to the position and surface orientation needed by a different pixel or a different frame (shift mappings were introduced as part of gradient domain rendering). For example a shift mapping called the reconnection shift makes it possible to reuse the portion of a path after the second surface intersection, only needing to trace one ray to check visibility between the first and second intersections. The reconnection shift is very effective for diffuse global illumination, and is the basis of a simplified algorithm called ReSTIR GI (which assumes all surfaces are Lambertian at the second intersection) that has been used in games.[48]: 22 [50]: 8 [51]

ReSTIR is primarily intended for real-time rendering, but it can also be used to speed up offline rendering. Although ReSTIR can be unbiased, potential drawbacks (which may make it less suitable for offline rendering) include unwanted correlation between errors in adjacent pixels (which may produce artifacts) and between frames (which may prevent convergence), and color noise (brightness of the pixels is correct, but the color has noise that does not decrease with multiple passes). To mitigate these problems, the temporal reuse portion can be omitted, and instead initial sampling and spatial reuse are repeated multiple times and the results are averaged.[49]: 12,16 

Many variations of ReSTIR now exist, applying the technique to volume rendering, and to depth of field and motion blur effects, and combining it with other techniques such as path guiding. Although most forms of ReSTIR work in screen space (with a reservoir for each pixel) there also exist world space versions that use RIS and spatiotemporal reuse to find good light samples for regions of space in a 3D grid.[52][53][54][55]

See also

[edit]

Notes

[edit]
  1. ^Kajiya, J. T. (1986). "The rendering equation". Proceedings of the 13th annual conference on Computer graphics and interactive techniques. ACM. CiteSeerX 10.1.1.63.1402.
  2. ^Lafortune, E, Mathematical Models and Monte Carlo Algorithms for Physically Based Rendering, (PhD thesis), 1996.
  3. ^Purcell, T J; Buck, I; Mark, W; and Hanrahan, P, "Ray Tracing on Programmable Graphics Hardware", Proc. SIGGRAPH 2002, 703 – 712. See also Purcell, T, Ray tracing on a stream processor (PhD thesis), 2004.
  4. ^Robison, Austin, "Interactive Ray Tracing on the GPU and NVIRT Overview", slide 37, I3D 2009.
  5. ^Vray demo; Other examples include Octane Render, Arion, and Luxrender.
  6. ^Seymour, Mike. "Disney's new Production Renderer 'Hyperion' – Yes, Disney!". fxguide. Retrieved 16 September 2017.
  7. ^Veach, E., and Guibas, L. J. Metropolis light transport. In SIGGRAPH’97 (August 1997), pp. 65–76.
  8. SmallPt is an educational path tracer by Kevin Beason. It uses 99 lines of C++ (including scene description). This page has a good set of examples of noise resulting from this technique.

References

[edit]
  1. ^ a b c d e f g h i j Kajiya, James T. (1986). "The rendering equation". ACM SIGGRAPH Computer Graphics. 20 (4): 143–150. doi:10.1145/15886.15902.
  2. ^ a b c d e f g h i j k l m Hughes, John F.; Van Dam, Andries; McGuire, Morgan; Sklar, David F.; Foley, James D.; Feiner, Steven K.; Akeley, Kurt (2014). Computer graphics : principles and practice (3rd ed.). Addison-Wesley. ISBN 978-0-321-39952-6.
  3. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "1.2 Photorealistic Rendering and the Ray-Tracing Algorithm". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 7 May 2026.
  4. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "13.1 The Light Transport Equation". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 7 May 2026.
  5. ^ a b Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "5.4 Film and Imaging". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  6. ^ a b c d e f g h i j k l m n Veach, Eric (1997). Robust Monte Carlo methods for light transport simulation (PDF) (PhD thesis). Stanford University.
  7. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "13.3 A Simple Path Tracer". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 5 May 2026.
  8. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "13.4 A Better Path Tracer". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 5 May 2026.
  9. ^ a b c Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "9.2 Diffuse Reflection". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 5 May 2026.
  10. ^ a b c d e f Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "2.2 Improving Efficiency". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  11. ^ a b Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "13.2 Path Tracing". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  12. ^ a b c Dutré, Philip; Bala, Kavita; Bekaert, Philippe (2015). Advanced Global Illumination (2nd ed.). A K Peters/CRC Press. ISBN 978-1-4987-8562-4.
  13. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8.2 Sampling and Integration". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  14. ^ a b Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8.5 Stratified Sampler". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  15. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8.6 Halton Sampler". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  16. ^ a b Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8.7 Sobol' Samplers". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  17. ^ Paulin, Lois; Bonneel, Nicolas; Coeurjolly, David; Iehl, Jean-Claude; Webanck, Antoine; Desbrun, Mathieu; Ostromoukhov, Victor (August 2020). "Sliced optimal transport sampling". ACM Transactions on Graphics. 39 (4). doi:10.1145/3386569.3392395.
  18. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8.1 Sampling Theory". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  19. ^ a b c Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "4.6 Color". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  20. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "7.2 Aggregates". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  21. ^ Stich, Martin (February 25, 2019). "Foreword". In Haines, Eric; Akenine-Möller, Tomas (eds.). Ray Tracing Gems: High-Quality and Real-Time Rendering with DXR and Other APIs. Berkeley, CA: Apress. doi:10.1007/978-1-4842-4427-2. ISBN 978-1-4842-4427-2.
  22. ^ a b Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "12.6 Light Sampling". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  23. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8.8 Image Reconstruction". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  24. ^ a b c d e Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "5 Film and Imaging: Further Reading". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  25. ^ a b c d Zwicker, M.; Jarosz, W.; Lehtinen, J.; Moon, B.; Ramamoorthi, R.; Rousselle, F.; Sen, P.; Soler, C.; Yoon, S.-E. (May 2015). "Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering". Computer Graphics Forum. 34 (2).
  26. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "8 Sampling and Reconstruction: Further Reading". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  27. ^ a b Marschner, Steve; Shirley, Peter (2022). Fundamentals of Computer Graphics (5th ed.). CRC Press. ISBN 978-1-003-05033-9.
  28. ^ Akenine-Möller, Tomas; Haines, Eric; Hoffman, Naty; Pesce, Angelo; Iwanicki, Michał; Hillaire, Sébastien (2018). Real-Time Rendering (4th ed.). Boca Raton, FL: A K Peters/CRC Press. ISBN 978-1138627000.
  29. ^ a b c d e f g Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "4.3 Surface Reflection". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 4 May 2026.
  30. ^ a b c d Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "9 Reflection Models". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  31. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "9 Reflection Models: Further Reading". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  32. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "10 Textures and Materials". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  33. ^ a b c d e Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "11.1 Volume Scattering Processes". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  34. ^ a b Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "11 Volume Scattering". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  35. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "11.2 Transmittance". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  36. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "11.4 Media". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 6 May 2026.
  37. ^ a b c d e f g Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (November 25, 2016). "16.4 Metropolis Light Transport". Physically Based Rendering: From Theory to Implementation (3rd ed.). Morgan Kaufmann. ISBN 978-0128006450. Retrieved 9 May 2026.
  38. ^ a b c d e f Veach, Eric; Guibas, Leonidas J. (1997). "Metropolis light transport". Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97. pp. 65–76. doi:10.1145/258734.258775. ISBN 0-89791-896-7.
  39. ^ a b c d e f Hachisuka, Toshiya; Kaplanyan, Anton S.; Dachsbacher, Carsten (27 July 2014). "Multiplexed metropolis light transport". ACM Transactions on Graphics. 33 (4): 1–10. doi:10.1145/2601097.2601138.
  40. ^ a b c Jensen, Henrik Wann (1996). "Global illumination using photon maps". Proceedings of the Eurographics Workshop on Rendering Techniques '96. Springer-Verlag. pp. 21–30. ISBN 3211828834.
  41. ^ a b c Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (November 25, 2016). "16.2 Stochastic Progressive Photon Mapping". Physically Based Rendering: From Theory to Implementation (3rd ed.). Morgan Kaufmann. ISBN 978-0128006450. Retrieved 29 April 2026.
  42. ^ Müller, Thomas; Gross, Markus; Novák, Jan (July 2017). "Practical Path Guiding for Efficient Light-Transport Simulation". Computer Graphics Forum. 36 (4): 91–100. doi:10.1111/cgf.13227.
  43. ^ a b c Georgiev, Iliyan; Křivánek, Jaroslav; Davidovič, Tomáš; Slusallek, Philipp (2012). "Light transport simulation with vertex connection and merging". ACM Transactions on Graphics. 31 (6): 1–10. doi:10.1145/2366145.2366211.
  44. ^ a b Hachisuka, Toshiya; Pantaleoni, Jacopo; Jensen, Henrik Wann (2012). "A path space extension for robust light transport simulation". ACM Transactions on Graphics. 31 (6): 1–10. doi:10.1145/2366145.2366210.
  45. ^ Pharr, Matt; Jakob, Wenzel; Humphreys, Greg (March 28, 2023). "13. Further Reading". Physically Based Rendering: From Theory to Implementation (4th ed.). Cambridge, Massachusetts: The MIT Press. ISBN 978-0262048026. Retrieved 29 April 2026.
  46. ^ a b Wyman, Chris (May 2025). I3D 2025 Chris Wyman's Keynote "ReSTIR: Traveling the Path of Reuse" (Speech). ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2025. Jersey City, NJ, USA.
  47. ^ a b c d e Bitterli, Benedikt; Wyman, Chris; Pharr, Matt; Shirley, Peter; Lefohn, Aaron; Jarosz, Wojciech (31 August 2020). "Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting" (PDF). ACM Transactions on Graphics. 39 (4). doi:10.1145/3386569.3392481.
  48. ^ a b c d e f g Wyman, Chris; Kettunen, Markus; Lin, Daqi; Bitterli, Benedikt; Yuksel, Cem; Jarosz, Wojciech; Kozlowski, Pawel; De Francesco, Giovanni (2023). "A Gentle Introduction to ReSTIR: Path Reuse in Real-time" (PDF). ACM SIGGRAPH 2023 Courses. pp. 1–38. doi:10.1145/3587423.3595511. ISBN 979-8-4007-0145-0.
  49. ^ a b Lin, Daqi; Kettunen, Markus; Bitterli, Benedikt; Pantaleoni, Jacopo; Yuksel, Cem; Wyman, Chris (2022). "Generalized Resampled Importance Sampling: Foundations of ReSTIR" (PDF). ACM Transactions on Graphics. 41 (4). doi:10.1145/3528223.3530158.
  50. ^ Ouyang, Yaobin; Liu, Shiqiu; Kettunen, Markus; Pharr, Matt; Pantaleoni, Jacopo (2021). "ReSTIR GI: Path Resampling for Real-Time Path Tracing". Computer Graphics Forum. 40 (8): 17–29. doi:10.1111/cgf.14378.
  51. ^ Knapik, Jakub; De Francesco, Giovanni; Zhdan, Dmitrii; Liu, Edward; Makarov, Evgeny; Kennedy, Jon; Marttila, Juho; Murphy, Michael; Hoobler, Nathan; Cheblokov, Tim; Kozlowski, Pawel (2024). "The Evolution of the Real-Time Lighting Pipeline in Cyberpunk 2077". GPU Zen 3: Advanced Rendering Techniques. Black Cat Publishing Inc. pp. 151–245. ISBN 979-8344236797.
  52. ^ Lin, Daqi; Wyman, Chris; Yuksel, Cem (2021). "Fast volume rendering with spatiotemporal reservoir resampling" (PDF). ACM Transactions on Graphics. 40 (6): 1–18. doi:10.1145/3478513.3480499.
  53. ^ Liu, Jeffrey; Lin, Daqi; Kettunen, Markus; Wyman, Chris; Ramamoorthi, Ravi (2025). "Reservoir Splatting for Temporal Path Resampling and Motion Blur" (PDF). Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers. pp. 1–11. doi:10.1145/3721238.3730646. ISBN 979-8-4007-1540-2.
  54. ^ Zeng, Zheng; Kettunen, Markus; Wyman, Chris; Wu, Lifan; Ramamoorthi, Ravi; Yan, Ling-Qi; Lin, Daqi (2025). "ReSTIR PG: Path Guiding with Spatiotemporally Resampled Paths". Proceedings of the SIGGRAPH Asia 2025 Conference Papers. pp. 1–11. doi:10.1145/3757377.3763813. ISBN 979-8-4007-2137-3.
  55. ^ Boksansky, Jakub; Jukarainen, Paula; Wyman, Chris (2021). "Rendering many lights with grid-based reservoirs". Ray Tracing Gems II. APress. pp. 351–365. doi:10.1007/978-1-4842-7185-8_23. ISBN 978-1484271841.