Skip to content

Using Vuforia Fusion Illumination

This article provides an overview of the Vuforia Fusion illumination and color correction featurese.

The Illumination Observer contains information about the estimated illumination characteristics of the user's physical environment computed by analyzing the live camera feed. Parameters of the Observation  represent a scene's illumination values and lets you render augmentations according to lighting changes of the real-world scene. An Illumination Observation can be returned from the State.

Create an Illumination Observer

Use the VuIlluminationConfig in your AR application to configure and create an Illumination Observer. The Observer is activated by default on creation.

1
2
3
4
5
6
VuIlluminationConfig illuminationConfig = vuIlluminationConfigDefault();

VuObserver* illuminationObserver = nullptr;
VuIlluminationCreationError creationError;

vuEnginecreateIlluminationObserver(mEngine, & illuminationObserver, &illuminationConfig, &creationError);

Observations

The Illumination Observer produces Observations that are collected in the State. Get the state from the Engine and parse them to the VuIlluminationObservationInfo.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
// Get all illumination observations from the state
vuObservationListCreate(&observationList);
vuStateGetIlluminationObservations(&state, observationList);
vuObservationListGetSize(observationList, &numObservations);

// Extract illumination information for scene rendering from the first illumination observation in the list
if (numObservations > 0)
{
    VuObservation* observation = nullptr;
    vuObservationListGetElement(observationList, 0, &observation);

    VuIlluminationObservationInfo mIlluminationInfo;
    vuIlluminationObservationGetInfo(observation, &mIlluminationInfo);

}

Destroy Observer and Observation

Destroy objects and processes after usage to free up memory.

For Illumination Observers, you call the following to destroy the observationList and Observer.

Destroy ObservationList

1
2
// Destroy observation list
vuObservationListDestroy(observationList);

Destroy the Observer

1
2
// Destroy the observer
vuObserverDestroy(illuminationObserver);

Illumination Values

ARKit and ARCore produce different illumination values. These values are different for a given lighting setting in the application's environment.  

  • ARKit generates two physical measurements:
    1. A color temperature measured in Kelvin.
    2. An intensity measured in Lumens. In applications such as Unity and SceneKit, these values are easily applied to light used in the rendering.  
  • ARCore supplies values that can be applied directly to a shader's RGB values, as well as an intensity value. 

For each platform, the values are returned in the following elements of the VuIlluminationObservationInfo:

Platform Elements
iOS ARKit ambientColorTemperature in Kelvin and ambientIntensity in Lumen.
Android ARCore intensityCorrection between 0.0 (black) and 1.0 (white) and colorCorrection in Vector4 containing RGBA values.

If the device does not support ARKit, then ambientIntensity and ambientColorTemperature returns VU_ILLUMINATION_AMBIENT_INTENSITY_UNAVAILABLE and VU_ILLUMINATION_AMBIENT_COLOR_TEMPERATURE_UNAVAILABLE.

The intensityCorrection and colorCorrection elements always return valid values that can be applied to a shader. If ARCore is not used, however, the returned values for intensityCorrection defaults to 0.466 (middle-grey) and the colorCorrection is set to 1.

Example Usage

The example code below demonstrates how to retrieve, convert, and load the rendered illumination values into the light shader for ARKit and ARCore enabled projects.

iOS ARKit

Retrieve the illumination values from the state and convert them to renderable values

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
// Factor in the intensity if available
    if (mIlluminationInfo.ambientIntensity != VU_ILLUMINATION_AMBIENT_INTENSITY_UNAVAILABLE)
    {
        // Divide by 1000 to normalize the ambient intensity value as it is in Lumens
        float intensity = mIlluminationInfo.ambientIntensity / 1000.0f;
        for (int i = 0; i < 3; i++)
        {
            colorValues.data[i] *= intensity;
            if (colorValues.data[i] > 1.0f)
            {
                colorValues.data[i] = 1.0f;
            }
        }
    } 
// Consider scene lighting lumens values between 200 - 1500
// 200 lumens is considered low brightness, 200 lumens in real life is similar to a dimmed light bulb;
// 1500 lumens is considered bright, in real life as bright as a standard light bulb.
// Higher values could be obtained from brighter sources

// Set the color if available
if (mIlluminationInfo.ambientColorTemperature != VU_ILLUMINATION_AMBIENT_COLOR_TEMPERATURE_UNAVAILABLE)
{
    VuVector3F rgbValues = convertKelvinToRGB(mIlluminationInfo.ambientColorTemperature);
    for (int i = 0; i < 3; i++)
    {
        colorValues.data[i] = rgbValues.data[i];
    }
}

Load the rendered values into your light shader algorithm and your lighting objects:

Android ARCore

Retrieve the illumination values from the State and convert them to renderable valuess:

1
2
3
4
5
6
7
8
9
// Use color correction if illumination information is available = 0,466f;
mIlluminationInfo.intensityCorrection = 0.466f;
mIlluminationInfo.colorCorrection = { 1.0f, 1.0f, 1.0f, 1.0f };

// Assumes we are rendering in gamma space, as per Google ARCore documentation
// https://developers.google.com/ar/reference/c/group/light#group__light_1ga70e0426f83e94a3f8f4c103a060b3414
// These correction values can be used in the shaders used for augmentations rendering: i.e. LightingShaders and DiffuseLightMaterials
     mIntensityCorrection = illumination.getIntensityCorrection() / 0.466f;
     mColorCorrection = illumination.getColorCorrection();

Load the rendered values into your light shader algorithm or your lighting objects in OpenGL.

Learn More

Vuforia Fusion

Vuforia Fusion Platform Handle

Observer and Observations