Post Pipeline
Post Pipeline
Embedded Files
The output of Predict Engine is a RAW image, similar to the one you could get by taking a photo with an actual camera. In order to display this image on screen, a post-process must be carried out, mostly in order to adjust the image dynamic range. This post-process is defined in the Post Pipeline section of the UVR Camera Settings component.
The post pipeline consists of the following main steps :
The post pipeline can be edited at runtime via the Interactive Settings interface.
White Balance
White Balance
When defining Still Cameras, white balance may be required. You can define the color that will be used as the reference white in the balance.
In the renderer view interface, an additional button next to the White Reference field enables you to pick a reference value directly in the Predict Engine render. Using this picker, the selected color will be the RGB value in the Predict Engine render before every post process (white balance included) : this option is especially useful if the current white reference is not white.
The white reference can be edited at runtime.
Denoiser
Denoiser
Path tracing can induce noise in the generated image. The longer an image takes to compute, the less noise there is. To reduce the noise while the image is rendering, a denoiser can be used.
The denoiser can be more or less agressive : its aggressivity is defined in the performance settings of Predict Engine.
The denoiser can be enabled/disabled at runtime.
When using the OIDN denoiser (default), Predict Engine must be installed at a path without any special character.
The OIDN denoiser is not available on GPU devices of type "Pascal".
Tone Mapper
Tone Mapper
Images produced by Predict Engine are defined within a dynamic range that cannot be displayed on the screen : values on screen must be contained within [0;255] whereas values in Predict Engine can go up to thousands or be contained in a much smaller interval. In order to adapt this dynamic range, we use a Tone Mapper.
Predict Engine supports different tone mappers that you can select in the Config field :
Linear tone mapper : a simple multiplicative factor is applied to the image. This factor is called the Exposure Value.
OCIO tone mapper : OpenColorIO (OCIO) is a color management solution based around a configuration file "config.ocio" that can be interprated by the OCIO library. The configuration file defines a list of color spaces with the corresponding conversions, as well as looks and views :
- A look is an artistic process to manage contrast, saturation, bleach effects, etc.
- A view transform is a display process to manage tone mapping, compression, gamma, etc.
- A view combines a view transform, a conversion from working color space to display color space, and an optional look.
An OCIO preset (or config) is defined by an input color space, a display color space, a view and an optionnal look. You can select the preset to use in the tone mapper Config combobox in the interface. In addition to the OCIO preset selection, you can define an Exposure value : a simple multiplicative factor applied to the image during the process (see details bellow).
Some presets (Standard and Human Vision) are always available to be used in the post pipeline. You can also define your own presets in the Tone Mapping Preferences (menu Predict Suite/Preferences... > Tone Mapping).
To define your custom presets, you can use the default OCIO config or change for a custom config file of your own. When changing the config file, make sure the XYZ space is properly selected, it is important for the color system management when using OCIO tone mapping in Predict Engine (see bellow). You can also select a default working space that will be used if it is not defined in the config.ocio file.
NB : The name of your custom preset defines what will show in the Config combobox in the interface.
The OCIO tone mapper works as follow (see figure bellow) :
The input data is given in the input color space in a given [min;max] range : the input color space is specified by the user,
The input data is then converted to a working color space : the working color space is usually defined in the config.ocio file but the user can override it or define a fallback value for files where the default value is not found,
The exposure is corrected with a linear factor (this is not defined in the config.ocio file),
Optionnal : if defined by the user, a user look is applied in addition to the look defined in the view,
The view is then applied : first the optionnal look, then the view transform, and finally the conversion from working color space to display color space,
The gamma correction is usually included in the view, but the user can choose to manually linearize/unlinearize the output at the end of the process (this is not defined in the config.ocio file),
The final output is the display data, defined in [0;1] in the display color space.
The OCIO tone mapper expects input data in a specific color space. When using an OCIO tone mapper, the color system gamut will be defined automatically to match the expected input color space. It is possible but not recommended to override this configuration. The gamma correction is always editable and it is interactive when using an OCIO tone mapper.
NB : this gamut is computed by making a conversion between the OCIO input color space and the reference XYZ space. The XYZ space must be defined in the config.ocio file and selected in the OCIO tone mapping Preferences.
Reinhard tone mapper (Expert mode) : a photographic tone mapper based on the Reinhard 2002 paper. The Reinhard 02 tone mapper is defined as follow :
Burn : the dodging-and-burning parameter, > 1 for burning (increasing the luminance) and < 1 for dodging (reducing the luminance),
Pre scale and Post scale : the scales used to compute the Reinhard parameters, see functions bellow.
AppyReinhard is the function that computes the tone mapped value from the input value. lPScale and lInvY2 are computed via the computeReinhardParameters function. Global inputs are : Average (the average of the entire image for each channel R, G and B), Burn, PreScale and PostScale.
Photographic tone mapper (Expert mode) : simulates the behaviour of an actual camera. The Photographic tone mapper is defined as follow :
Exposure : the exposure time in seconds (can be edited at run time and automatized using the Auto Exposure option),
Aperture : the aperture of the optical system, the higher the aperture the more light enters the system (can be edited at run time),
ISO : the sensitivity of the sensor, the higher the sensitivity the less light is needed to expose the sensor (can be edited at run time),
Gain RGB : a multiplicative factor applied to the final output,
Bit Depth : the number of bits each pixel is stored on,
Pixel Offset : the minimum value taken by a pixel, the value should be in the interval [0; 2^BitDepth - 1].
If any of the following values {exposure time, ISO, Gain RGB} are set to 0, the output will be a completely black image.
The exposure of the tone mapper (Linear : Exposure Value, OCIO : Exposure Value, Reinhard : Burn, Photographic : Exposure) can be chosen manually or it can be computed automatically (when the Auto Exposure field is enabled). When the auto exposure mode is enabled, the exposure value is computed so that the average of a zone at the center of the image is around 128 (half the maximum value). The EV +/- field enables you to make the image darker or brighter by changing this expected value.
The tone mapper settings can be edited at run time.
Fireflies management
Fireflies management
Fireflies are rendering artifacts resulting from numerical instabilities in solving the rendering equation. They manifest themselves as anomalously-bright single pixels scattered over parts of the image.
To limit the impact of these incoherences, we replace abnormal values by the mean of their neighbours. Values are identified as abnormals when they are higher than : [mean of the neighbours + threshold * neighbours standard deviation]. If the Expert Mode is enabled, you can define this threshold.
False Colors
False Colors
When using a False Color Color System, the raw output image from Predict Engine is visualized as a representation of the measured physical quantity in false colors.
Luminancemeter sensors output a value in cd/m² : the luminance or Y component of the CIE XYZ color.
Spectroradiometer sensors output a value in W/m²/sr. You can define which channel is displayed.
If the Auto Range is enabled, the color map will represent all the values in the image. If it is disabled, a "Range" field enables you to define the range of values (in % or with given values) that should be represented. This enables you to ignore the lowest/highest values in the image that may correspond to fireflies/noise or saturated zones with direct illumination.
You can choose to display the value with different scales (see Figure) :
linear,
logarithm,
square root,
cubic root,
You can choose to show the scale on screen or not.
Figure : impact of the false color scale in the [0;1] range
Polarimeter sensors are necessarily in false colors as the raw output image from Predict Engine cannot be interpreted as an RGB visualisation of the scene. Six visualization modes are available for the polarization in addition to the four Stokes components :
The Degree mode displays the degree of polarization in the range [0;1] using a red scale colormap (see Figure B bellow),
The Orientation mode displays the orientation of the polarized light, in the range [0;180] degrees, using a Rainbow colormap (see Figure D bellow),
The Ellipticity mode displays the ellipticity of the circularly polarized light, in the range [-45;45] degrees : an ellipticity of +45° (right) is represented by the red color, an ellipticity of -45° (left) is represented by the blue color, and an ellipticity of 0° is represented by the black color (see Figure F bellow),
The Type mode displays the type of the polarization : linear polarization is represented by the cyan color and circular polarization is represented by the yellow color (see Figure C bellow),
The Chirality mode displays the chirality of the circularly polarized light (S3 component of the Stokes vector) : a right chirality is represented by the blue color and a left chirality is represented by the yellow color (see Figure E bellow),
The Plane mode displays the orientation of the linearly polarized light (circularly or elliptically polarized light will be ignored), in [0;180] degrees, using a red/green colormap for the S1 Stokes component and a blue/yellow colormap for the S2 Stokes component (see Figure G bellow),
The S0 mode display the raw S0 component of the Stokes vector,
The S1 Reduced, S2 Reduced, and S3 Reduced modes display the reduced components of the Stokes vector : the raw component divided by S0.
More details on the visualisation modes of the polarization :
Wilkie, A. and Weidlich, A. (2010). A standardised polarisation visualisation for images. In Proceedings of the 26th Spring Conference on Computer Graphics, pages 43–50. ACM.
The false color settings can be edited at run time.
Example scene defined with three spheres (one red diffuse, one metallic and one glass), a mirror in the background, and two polarizers on the front (linear polarizer on the left, circular polarizer on the right), lit by a D65 ambient light.
Figure A : scene rendered with a still camera sensor
Figure B : scene rendered in the Degree mode.
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear/circular polarizer is purely linearly/circularly polarized.
The diffuse material is not inducing polarization.
Figure C : scene rendered in the Type mode
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear/circular polarizer is purely linearly/circularly polarized.
The diffuse material is not inducing polarization.
Figure D : scene rendered in the Orientation mode.
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear polarizer is purely linearly polarized in a single direction.
Figure E : scene rendered in the Chirality mode.
The light going through the circular polarizer is purely circularly polarized with one chirality.
Figure F : scene rendered in the Ellipticity mode.
The light going through the circular polarizer is purely circularly polarized with one chirality.
Figure G : scene rendered in the Plane mode
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear polarizer is purely linearly polarized in a single direction.
Figure H : scene rendered in the S0 mode.
Figure I : scene rendered in the S1 Reduced mode.
Figure J : scene rendered in the S2 Reduced mode.
Figure K : scene rendered in the S3 Reduced mode
Custom post pipeline
Custom post pipeline
When using a custom sensor, you can define manually the output of the Predict Engine renderer. The type of the color system impacts the definition of the Post Pipeline. Three types of color systems are available :
RGB : the output is defined by the three given XYZ channels, the raw image is then converted to LDR RGB using a given gamut, a gamma correction is applied.
If the photometric preset is used on the sensor, the channels can be defined by the default Photometric XYZ channels.
Otherwise, the channels are selected among the available layers (see section bellow) and channels defined on the sensor.
Raw : the output is defined by the three given RGB channels, the raw image is only processed with a gamma correction.
If the photographic preset is used on the sensor, the channels can be defined by the default Photographic RGB channels.
Otherwise, the channels are selected among the available layers (see section bellow) and channels defined on the sensor.
False Color : one of the sensor channels is visualized as a representation of the measured physical quantity in false colors. It is also possible to display :
the normals/tangents/bi-tangents of the scene : each vector (x,y,z) is represented as a color (r,g,b),
the material : each ID is represented by a color, the color are regularly selected in the given colormap,
the depth : the distance between the camera and the elements in the scene is displayed using the given colormap.
Page updated
Report abuse