Skip to content

Vuforia Engine Lifecycle in Native

This page shows a step-by-step introduction to configuring and managing the Engine and application lifecycle.

ng-10-api-engine

The general workflow for Vuforia Engine may look like this:

  • Configure and create Engine
  • Configure and create Observer
  • Parse the State and retrieve Observations
  • Render augmentations based on Observations

Manage Engine Lifecycle

All Vuforia applications begin by creating and starting the Vuforia Engine. Only one Engine instance can be created at a time. Stop and destroy the Engine before creating another.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
VuEngine* engine = NULL; 

// Create an engine instance using the default parameters 
vuEngineCreate(&engine, NULL, NULL); 

// Start the engine 
vuEngineStart(engine); 

// Stop the engine 
vuEngineStop(engine); 

// Destroy the engine 
vuEngineDestroy(engine); 

Always ensure that create methods are concluded with a destroy call to free up memory. Similarly, acquire methods need to be accompanied by a release call.

Add the Vuforia Engine lifecycle to your mobile lifecycle similar to this: 

  • Initialize and create app: Configure and create vuEngineCreate
  • Start, resume app and execute primary app logic: Add additional configurations and start vuEngineStart
  • During pause and background app logic: Stop vuEngineStop
  • Deinitialize and free memory in app: Destroy vuEngineDestroy

Permissions

Your App must fulfill platform-specific permission and configuration requirements in order to successfully create an Engine instance.

Your App is responsible for executing the following platform-specific actions prior to invoking vuEngineCreate(), otherwise the Engine creation call may fail or Engine may not work properly:

Android
  • Camera permission: Add the "android.permission.CAMERA" permission to the AndroidManifest.xml and request it at runtime in your App's code. See the Vuforia Native Android Sample’s VuforiaActivity.kt for an example.
  • Network-related permissions: Add the "android.permission.INTERNET" and "android.permission.ACCESS_NETWORK_STATE" permissions to allow Vuforia Engine to leverage various cloud services and to ensure an optimal AR experience on all devices by downloading device-specific Engine settings.

If these permissions are not granted, Vuforia Engine will fail to initialize with the error VU_ENGINE_CREATION_ERROR_PERMISSION_ERROR.

  • High sensor sampling rate: When targeting Android 12 or above, add the "android.permission.HIGH_SAMPLING_RATE_SENSORS" permission to the AndroidManifest.xml to allow Engine to receive IMU sensor readings at a high rate, otherwise device tracking may not work correctly.

iOS
  • Camera usage description: Add the key "NSCameraUsageDescription" to the Info.plist with an explanation why your App requires camera access, otherwise the App will be rejected by the App Store submission process.
  • Camera access: Request access to the camera by calling the "AVCaptureDevice requestAccessForMediaType" API with the media type "AVMediaTypeVideo" and request it at runtime in your App’s code, otherwise Vuforia Engine will fail to initialize with the error VU_ENGINE_CREATION_ERROR_PERMISSION_ERROR. See Vuforia Native iOS sample’s VuforiaViewController.swift for an example.

UWP
  • Video capturing and network device capability: Add the following snippet to the <Capabilities> tag in your App's Package.appxmanifest to enable camera capturing and full network capabilities, otherwise Engine will not work properly:
<Capabilities>
    <Capability Name="internetClient" />
    <DeviceCapability Name="webcam" />
</Capabilities>

Lumin
  • Camera permissions: Add "CameraCapture" and "ComputerVision" permissions to the manifest.xml.
  • Network-related permissions: Add the "Internet" permission to allow Vuforia Engine to leverage various cloud services and ensure an optimal AR experience on all devices by downloading device-specific Engine settings.
  • Perception system permissions: Add "LowLatencyLightwear", "PcfRead" and "WorldReconstruction" permissions to the manifest.xml.

If permissions are not granted, Vuforia Engine will fail to initialize with the error VU_ENGINE_CREATION_ERROR_PERMISSION_ERROR.

Configure the Engine and acquire the State

Add basic configurations to the Engine such as a Vuforia Developer license key. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
void basic_engine_config() 
 { 
     // Create a configuration set 
     VuEngineConfigSet* configSet = NULL; 

     vuEngineConfigSetCreate(&configSet); 

     // Create the license configuration element 
     VuLicenseConfig licenseConfig = { 0 }; 
     licenseConfig.key = "1234567890"; 

     // Add the license configuration to the set 
     vuEngineConfigSetAddLicenseConfig(configSet, &licenseConfig); 

     vuEngineCreate(&engine, configSet, NULL);

     // Add more configuration elements 
     // ... 
 } 

Similar to Engine, destroy the configuration when finished with the AR session (by calling vuEngineConfigSetDestroy(configSet)). 

Acquire State

After you have started the Engine, you can start to pull updates from the state: 

1
2
3
// Get update from Engine via pull mechanism: get latest state 
VuState* state = NULL; 
vuEngineAcquireLatestState(engine, &state); 

And when done,  

1
2
// Release the state 
vuStateRelease(state); 

Or alternatively, get updates with a push mechanism: 

1
2
// Get update from Engine via push mechanism: register a callback 
vuEngineRegisterStateHandler(engine, myHandler, NULL); 

Where myHandler is a method in which the state is retrieved and contained. A StateHandler can be unregistered by providing a NULL as the second parameter to the method. The third parameter is the client data that you pass with the state updates.  

1
vuEngineRegisterStateHandler(engine, NULL, NULL); 

NOTE: The callack that you register is running in the camera thread. You should therefore not assign heavy processes in this method. The callback should only be used for doing critical operations such as Computer Vision tasks and should not process tasks such as UI updates. Please refer to the Vuforia Native Samples that demonstrates best practice on thread handling. 

Create an Observer

An Observer is created with a configuration structure designated to the Observer type. In most cases, this includes a databasePath and a targetName from a database.  By default an Observer is automatically activated (i.e. will start processing after creation). In this example, we create a simple Observer for an Image Target: 

1
2
3
4
5
6
7
8
// Create an image target config 
VuImageTargetConfig imageTargetConfig = vuImageTargetConfigDefault(); 
imageTargetConfig.databasePath = "StonesAndChips.xml"; 
imageTargetConfig.targetName = "stones"; 

// Create an Image Target Observer 
VuObserver* imageTargetObserver = NULL; 
vuEngineCreateImageTargetObserver(engine, &imageTargetObserver, &imageTargetConfig, NULL); 

Add more configurations on your own initiative to set its scale, pose offset or control whether it should be activated upon creation. For other target types, more configurations may be necessary to add. If you significantly modify Observers via setters, which share the same database, it is generally recommended to deactivate the Observers during this operation for performance reasons. You can do that either by deactivating the activation in the Observer configuration or by calling vuObserverDeactivate(observer)/vuObserverActivate(observer). Refer to the native API overviews of each Vuforia feature to learn more.  

Since we have created the Observer for an Image Target, we also need to destroy it at the end of the AR session. 

1
2
// Destroy the observer 
vuObserverDestroy(imageTargetObserver); 

Parse the State and Retrieve Observations

For our Image Target Observer, we can now retrieve information about its status by requesting it from the state by using the following function: 

1
2
3
4
5
6
// Create observation list
VuObservationList* obsList = NULL; 
vuObservationListCreate(&obsList); 

// Get and parse image target observations list 
 vuStateGetImageTargetObservations(state,obsList); 

Or, get the Observation directly from the Observer which is particular suitable for updating Observer individually which may also be more manageable.  

1
2
// Get and parse image target observations list from the
vuStateGetObservationsByObserver(state, imageTargetObserver, obsList); 

You can then parse the list by getting the list size and retrieve each element via a get method,  vuObservationListGetElement. From there you can parse all the info from your observation, such as pose info, target info (or status info).  

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
int32_t listSize = 0;
vuObservationListGetSize(obsList, &listSize);

// Parse the image target list
     for (int i = 0; i < listSize; i++)
     {
             VuObservation* obs = NULL;
             vuObservationListGetElement(obsList, i, &obs);

             VuImageTargetObservationTargetInfo targetInfo;
             vuImageTargetObservationGetTargetInfo(obs, &targetInfo);

             VuImageTargetObservationStatusInfo statusInfo;
             vuImageTargetObservationGetStatusInfo(obs, &statusInfo);

             VuPoseInfo poseInfo;
             vuObservationGetPoseInfo(obs, &poseInfo);

             if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE)
             {
                     // Do something with poseInfo and targetInfo
             }
         }

As an alternative, get all observations from the state and parse them type by type.  

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// Get update from Engine via pull mechanism: get latest state
VuState* state = NULL;
vuEngineAcquireLatestState(engine, &state);

// Parse the state

// Create observation list
VuObservationList* obsList = NULL;
vuObservationListCreate(&obsList);
vuStateGetObservations(state, obsList);
int32_t listSize = 0;
vuObservationListGetSize(obsList, &listSize);

// Parse the list
for (int i = 0; i < listSize; i++)
     {
         VuObservation* obs = NULL;
         vuObservationListGetElement(obsList, i, &obs);

         if (vuObservationIsType(obs, VU_OBSERVATION_IMAGE_TARGET_TYPE) == VU_TRUE)
         {
                 VuImageTargetObservationTargetInfo targetInfo;
                 vuImageTargetObservationGetTargetInfo(obs, &targetInfo);

                 VuPoseInfo poseInfo;
                 vuObservationGetPoseInfo(obs, &poseInfo);    

                 if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE)
                 {
                         // Do something with poseInfo and targetInfo
                 
         }
         // Else if ..
     }

Again, make sure to free up the memory by destroying the created vuObservationList. 

1
2
3
4
5
// Destroy observation list
vuObservationListDestroy(obsList);

// Release the state
vuStateRelease(state);

You can find more information in Observer and Observations.

Render Content Based on Status

Use the retrieved pose from the Observation of an Observer and render content in relation to the pose information of the target by using the RenderState. The RenderState is a component from the state that provides access to the projection matrix, view matrix, viewport, and background video mesh. See Rendering in Native for more information.  

Based on the observation statuses, a cube is rendered with the VuRenderState with respect to the pose information. The code example below is the assembly of the snippets from the above steps. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
void basic_image_target_update_render()
{
     VuEngine* engine = NULL;

     // Create an engine using the default parameters
     vuEngineCreate(&engine, NULL, NULL);

     // Create an image target config
     VuImageTargetConfig imageTargetConfig = vuImageTargetConfigDefault();
     imageTargetConfig.databasePath = "StonesAndChips.xml";
     imageTargetConfig.targetName = "stones";

     // Create an Image Target Observer
     VuObserver* imageTargetObserver = NULL;
     vuEngineCreateImageTargetObserver(engine, &imageTargetObserver, imageTargetConfig, NULL);

     // Start the engine
     vuEngineStart(engine);

     // Get update from Engine via pull mechanism: get latest state
     VuState* state = NULL;
     vuEngineAcquireLatestState(engine, &state);

     // Parse the state

     // Get Render State
     VuRenderState renderState;
     vuStateGetRenderState(state, &renderState);

     // Create observation list
     VuObservationList* obsList = NULL;
     vuObservationListCreate(&obsList);
     vuStateGetObservations(state, obsList);
     int32_t listSize = 0;
     vuObservationListGetSize(obsList, &listSize);

     // Parse the it list
     for (int i = 0; i < listSize; i++)
     {
         VuObservation* obs = NULL;
         vuObservationListGetElement(obsList, i, &obs);

         VuImageTargetObservationTargetInfo targetInfo;
         vuImageTargetObservationGetTargetInfo(obs, &targetInfo);

         VuPoseInfo poseInfo;
         vuObservationGetPoseInfo(obs, &poseInfo);

         if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE)
         {
             renderCube(renderState.projectionMatrix, poseInfo.pose,
                             vuVector2FToVector3F(targetInfo.size));
         }
     }
         //else if ..

     // Destroy observation list
     vuObservationListDestroy(obsList);

     // Release the state
     vuStateRelease(state);

     // Destroy the observer
     vuObserverDestroy(imageTargetObserver);

     // Stop the engine
     vuEngineStop(engine);

     // Destroy the engine
     vuEngineDestroy(engine);
}

void renderCube(VuMatrix44F projMatrix, VuMatrix44F poseMatrix, VuVector3F scale)
 {
     //
 }

Error Handling

The Vuforia Engine API promotes the implementation of proper error handling in your application. This applies, for example, to creation and configuration calls. You would, therefore, need to add error handling in the above code examples to Engine creation, adding a license, Observer creation, and creation of an Observation list. The Vuforia Engine API has various error codes specific to the Observer, Engine configuration, and platform, which are helpful in validation activities during the development process and end usage. Please refer to the API reference library for the complete list of error codes. See also Troubleshooting your App for more debugging instructions.

Error management example for Engine creation: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
VuEngine* engine = NULL;

VuErrorCode error;

// Create an engine using default parameters
if (vuEngineCreate(&engine, NULL, &error) != VU_SUCCESS)
{
    VU_LOG_ERROR("failed to create a vuforia engine", error);
    return -1;
}

Error management example for Observer creation: 

1
2
3
4
5
6
7
8
VuImageTargetCreationError imageTargetCreationError;

// Create Image Target Observer
if (vuEngineCreateImageTargetObserver(engine, &imageTargetObserver, imageTargetConfig, &imageTargetCreationError) != VU_SUCCESS)
{
        LOG("Error creating image target observer: 0x%02x", imageTargetCreationError);
    return -1;
}

Next Steps

Observer and Observations

Status Poses and Status Info

Vuforia Features

Vuforia Feature Overview