Vuforia Engine Lifecycle in Native

This page shows a step-by-step introduction to configuring and managing the Engine and application lifecycle.

The general workflow for Vuforia Engine may look like this:

  • Configure and create Engine
  • Configure and create Observer
  • Parse the State and retrieve Observations
  • Render augmentations based on Observations

Manage Engine Lifecycle

All Vuforia applications begin by creating and starting the Vuforia Engine. Only one Engine instance can be created at a time. Stop and destroy the Engine before creating another.

12345678910111213
Copy
VuEngine* engine = NULL; // Create an engine instance using the default parameters vuEngineCreate(&engine, NULL, NULL); // Start the engine vuEngineStart(engine); // Stop the engine vuEngineStop(engine); // Destroy the engine vuEngineDestroy(engine);

Always ensure that create methods are concluded with a destroy call to free up memory. Similarly, acquire methods need to be accompanied by a release call.

Add the Vuforia Engine lifecycle to your mobile lifecycle similar to this: 

  • Initialize and create app: Configure and create vuEngineCreate
  • Start, resume app and execute primary app logic: Add additional configurations and start vuEngineStart
  • During pause and background app logic: Stop vuEngineStop
  • Deinitialize and free memory in app: Destroy vuEngineDestroy

Permissions

Your App must fulfill platform-specific permission and configuration requirements in order to successfully create an Engine instance.

Your App is responsible for executing the following platform-specific actions prior to invoking vuEngineCreate(), otherwise the Engine creation call may fail or Engine may not work properly:

Android

  • Camera permission: Add the "android.permission.CAMERA" permission to the AndroidManifest.xml and request it at runtime in your App's code. See the Vuforia Native Android Sample’s VuforiaActivity.kt for an example.
  • Network-related permissions: Add the "android.permission.INTERNET" and "android.permission.ACCESS_NETWORK_STATE" permissions to allow Vuforia Engine to leverage various cloud services and to ensure an optimal AR experience on all devices by downloading device-specific Engine settings.

If these permissions are not granted, Vuforia Engine will fail to initialize with the error VU_ENGINE_CREATION_ERROR_PERMISSION_ERROR.

  • High sensor sampling rate: When targeting Android 12 or above, add the "android.permission.HIGH_SAMPLING_RATE_SENSORS" permission to the AndroidManifest.xml to allow Engine to receive IMU sensor readings at a high rate, otherwise device tracking may not work correctly.

iOS

  • Camera usage description: Add the key "NSCameraUsageDescription" to the Info.plist with an explanation why your App requires camera access, otherwise the App will be rejected by the App Store submission process.
  • Camera access: Request access to the camera by calling the "AVCaptureDevice requestAccessForMediaType" API with the media type "AVMediaTypeVideo" and request it at runtime in your App’s code, otherwise Vuforia Engine will fail to initialize with the error VU_ENGINE_CREATION_ERROR_PERMISSION_ERROR. See Vuforia Native iOS sample’s VuforiaViewController.swift for an example.

UWP

  • Video capturing and network device capability: Add the following snippet to the <Capabilities> tag in your App's Package.appxmanifest to enable camera capturing and full network capabilities, otherwise Engine will not work properly:
1234
Copy
<Capabilities> <Capability Name="internetClient" /> <DeviceCapability Name="webcam" /> </Capabilities>

Lumin

  • Camera permissions: Add "CameraCapture" and "ComputerVision" permissions to the manifest.xml.
  • Network-related permissions: Add the "Internet" permission to allow Vuforia Engine to leverage various cloud services and ensure an optimal AR experience on all devices by downloading device-specific Engine settings.
  • Perception system permissions: Add "LowLatencyLightwear", "PcfRead" and "WorldReconstruction" permissions to the manifest.xml.

If permissions are not granted, Vuforia Engine will fail to initialize with the error VU_ENGINE_CREATION_ERROR_PERMISSION_ERROR.

Configure the Engine and acquire the State 

Add basic configurations to the Engine such as a Vuforia Developer license key. 

12345678910111213141516171819
Copy
void basic_engine_config() { // Create a configuration set VuEngineConfigSet* configSet = NULL; vuEngineConfigSetCreate(&configSet); // Create the license configuration element VuLicenseConfig licenseConfig = { 0 }; licenseConfig.key = "1234567890"; // Add the license configuration to the set vuEngineConfigSetAddLicenseConfig(configSet, &licenseConfig); vuEngineCreate(&engine, configSet, NULL); // Add more configuration elements // ... }

Similar to Engine, destroy the configuration when finished with the AR session (by calling vuEngineConfigSetDestroy(configSet)). 

Acquire State 

After you have started the Engine, you can start to pull updates from the state: 

123
Copy
// Get update from Engine via pull mechanism: get latest state VuState* state = NULL; vuEngineAcquireLatestState(engine, &state);

And when done,  

12
Copy
// Release the state vuStateRelease(state);

Or alternatively, get updates with a push mechanism: 

12
Copy
// Get update from Engine via push mechanism: register a callback vuEngineRegisterStateHandler(engine, myHandler, NULL);

Where myHandler is a method in which the state is retrieved and contained. A StateHandler can be unregistered by providing a NULL as the second parameter to the method. The third parameter is the client data that you pass with the state updates.  

1
Copy
vuEngineRegisterStateHandler(engine, NULL, NULL);

NOTE: The callack that you register is running in the camera thread. You should therefore not assign heavy processes in this method. The callback should only be used for doing critical operations such as Computer Vision tasks and should not process tasks such as UI updates. Please refer to the Vuforia Native Samples that demonstrates best practice on thread handling. 

Create an Observer 

An Observer is created with a configuration structure designated to the Observer type. In most cases, this includes a databasePath and a targetName from a database.  By default an Observer is automatically activated (i.e. will start processing after creation). In this example, we create a simple Observer for an Image Target: 

12345678
Copy
// Create an image target config VuImageTargetConfig imageTargetConfig = vuImageTargetConfigDefault(); imageTargetConfig.databasePath = "StonesAndChips.xml"; imageTargetConfig.targetName = "stones"; // Create an Image Target Observer VuObserver* imageTargetObserver = NULL; vuEngineCreateImageTargetObserver(engine, &imageTargetObserver, &imageTargetConfig, NULL);

Add more configurations on your own initiative to set its scale, pose offset or control whether it should be activated upon creation. For other target types, more configurations may be necessary to add. If you significantly modify Observers via setters, which share the same database, it is generally recommended to deactivate the Observers during this operation for performance reasons. You can do that either by deactivating the activation in the Observer configuration or by calling vuObserverDeactivate(observer)/vuObserverActivate(observer). Refer to the native API overviews of each Vuforia feature to learn more.  

Since we have created the Observer for an Image Target, we also need to destroy it at the end of the AR session. 

12
Copy
// Destroy the observer vuObserverDestroy(imageTargetObserver);

Parse the State and Retrieve Observations 

For our Image Target Observer, we can now retrieve information about its status by requesting it from the state by using the following function: 

123456
Copy
// Create observation list VuObservationList* obsList = NULL; vuObservationListCreate(&obsList); // Get and parse image target observations list vuStateGetImageTargetObservations(state,obsList);

Or, get the Observation directly from the Observer which is particular suitable for updating Observer individually which may also be more manageable.  

12
Copy
// Get and parse image target observations list from the vuStateGetObservationsByObserver(state, imageTargetObserver, obsList);

You can then parse the list by getting the list size and retrieve each element via a get method,  vuObservationListGetElement. From there you can parse all the info from your observation, such as pose info, target info (or status info).  

123456789101112131415161718192021222324
Copy
int32_t listSize = 0; vuObservationListGetSize(obsList, &listSize); // Parse the image target list for (int i = 0; i < listSize; i++) { VuObservation* obs = NULL; vuObservationListGetElement(obsList, i, &obs); VuImageTargetObservationTargetInfo targetInfo; vuImageTargetObservationGetTargetInfo(obs, &targetInfo); VuImageTargetObservationStatusInfo statusInfo; vuImageTargetObservationGetStatusInfo(obs, &statusInfo); VuPoseInfo poseInfo; vuObservationGetPoseInfo(obs, &poseInfo); if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE) { // Do something with poseInfo and targetInfo } }

As an alternative, get all observations from the state and parse them type by type.  

1234567891011121314151617181920212223242526272829303132333435
Copy
// Get update from Engine via pull mechanism: get latest state VuState* state = NULL; vuEngineAcquireLatestState(engine, &state); // Parse the state // Create observation list VuObservationList* obsList = NULL; vuObservationListCreate(&obsList); vuStateGetObservations(state, obsList); int32_t listSize = 0; vuObservationListGetSize(obsList, &listSize); // Parse the list for (int i = 0; i < listSize; i++) { VuObservation* obs = NULL; vuObservationListGetElement(obsList, i, &obs); if (vuObservationIsType(obs, VU_OBSERVATION_IMAGE_TARGET_TYPE) == VU_TRUE) { VuImageTargetObservationTargetInfo targetInfo; vuImageTargetObservationGetTargetInfo(obs, &targetInfo); VuPoseInfo poseInfo; vuObservationGetPoseInfo(obs, &poseInfo); if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE) { // Do something with poseInfo and targetInfo } // Else if .. }

Again, make sure to free up the memory by destroying the created vuObservationList

12345
Copy
// Destroy observation list vuObservationListDestroy(obsList); // Release the state vuStateRelease(state);

You can find more information in Observer and Observations.

Render Content Based on Status 

Use the retrieved pose from the Observation of an Observer and render content in relation to the pose information of the target by using the RenderState. The RenderState is a component from the state that provides access to the projection matrix, view matrix, viewport, and background video mesh. See Rendering in Native for more information.  

Based on the observation statuses, a cube is rendered with the VuRenderState with respect to the pose information. The code example below is the assembly of the snippets from the above steps. 

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677
Copy
void basic_image_target_update_render() { VuEngine* engine = NULL; // Create an engine using the default parameters vuEngineCreate(&engine, NULL, NULL); // Create an image target config VuImageTargetConfig imageTargetConfig = vuImageTargetConfigDefault(); imageTargetConfig.databasePath = "StonesAndChips.xml"; imageTargetConfig.targetName = "stones"; // Create an Image Target Observer VuObserver* imageTargetObserver = NULL; vuEngineCreateImageTargetObserver(engine, &imageTargetObserver, imageTargetConfig, NULL); // Start the engine vuEngineStart(engine); // Get update from Engine via pull mechanism: get latest state VuState* state = NULL; vuEngineAcquireLatestState(engine, &state); // Parse the state // Get Render State VuRenderState renderState; vuStateGetRenderState(state, &renderState); // Create observation list VuObservationList* obsList = NULL; vuObservationListCreate(&obsList); vuStateGetObservations(state, obsList); int32_t listSize = 0; vuObservationListGetSize(obsList, &listSize); // Parse the it list for (int i = 0; i < listSize; i++) { VuObservation* obs = NULL; vuObservationListGetElement(obsList, i, &obs); VuImageTargetObservationTargetInfo targetInfo; vuImageTargetObservationGetTargetInfo(obs, &targetInfo); VuPoseInfo poseInfo; vuObservationGetPoseInfo(obs, &poseInfo); if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE) { renderCube(renderState.projectionMatrix, poseInfo.pose, vuVector2FToVector3F(targetInfo.size)); } } //else if .. // Destroy observation list vuObservationListDestroy(obsList); // Release the state vuStateRelease(state); // Destroy the observer vuObserverDestroy(imageTargetObserver); // Stop the engine vuEngineStop(engine); // Destroy the engine vuEngineDestroy(engine); } void renderCube(VuMatrix44F projMatrix, VuMatrix44F poseMatrix, VuVector3F scale) { // }

Error Handling 

The Vuforia Engine API promotes the implementation of proper error handling in your application. This applies, for example, to creation and configuration calls. You would, therefore, need to add error handling in the above code examples to Engine creation, adding a license, Observer creation, and creation of an Observation list. The Vuforia Engine API has various error codes specific to the Observer, Engine configuration, and platform, which are helpful in validation activities during the development process and end usage. Please refer to the API reference library for the complete list of error codes. See also Troubleshooting your App for more debugging instructions.

Error management example for Engine creation: 

1234567891011
Copy
VuEngine* engine = NULL; VuErrorCode error; // Create an engine using default parameters if (vuEngineCreate(&engine, NULL, &error) != VU_SUCCESS) { VU_LOG_ERROR("failed to create a vuforia engine", error); return -1; }

Error management example for Observer creation: 

12345678
Copy
VuImageTargetCreationError imageTargetCreationError; // Create Image Target Observer if (vuEngineCreateImageTargetObserver(engine, &imageTargetObserver, imageTargetConfig, &imageTargetCreationError) != VU_SUCCESS) { LOG("Error creating image target observer: 0x%02x", imageTargetCreationError); return -1; }

Next Steps 

Observer and Observations 

Status Poses and Status Info 

Vuforia Features 

Vuforia Feature Overview 

Can this page be better?
Share your feedback via our issue tracker