Model Targets API Overview

This article describes the native API for configuring Model Targets and Advanced Model Targets, their Guide Views, and the available modes that help improve Model Target tracking experiences.

For a general introduction to the core concepts of the Vuforia Engine API, please see the Vuforia Engine 10 API. Please see the Model Targets main page for an introduction to Model Targets and Advanced Model Targets. We also encourage you to try the samples that you are free to use as a foundation for your own AR application. See Native Samples for more information.

A Model Target is tracked by creating an Observer from a Model Target Database. The setup of Model Targets is similar to other Observers but with some distinct differences:

  • Model Targets are set up with Guide View images that help the user approach the object from a point where it can be detected, and tracking is then started.
  • Advanced Model Targets are set up with Advanced Views and trained so objects are automatically detected and tracked.

General Setup

Please see the Engine Lifecycle for a general introduction to the Vuforia Engine lifecycle. The Vuforia Engine is required to create and use Observers.

Loading a Model Target Database

Prepare and create Model Targets from 3D models with the Model Target Generator (MTG) and generate Model Target databases as pairs of XML and DAT dataset files.

Model Target Database

When loading Model Target databases, consider the following:

  • Multiple Model Target Observers can be created simultaneously, subject to memory usage limits.
  • Only one Model Target Observer may be active at any one time. (This restriction does not apply to tracking alongside other kinds of observer types. For example, you may have multiple active Image Target Observers at the same time as you have an active Model Target Observer).
  • For Advanced Model Target databases with multiple Model Targets, you may configure Model Target Observers for each available target in the Advanced Model Target database. Vuforia Engine will automatically detect and set the currently visible object as the active Model Target. For more information, see the section Controlling Recognition Behavior for Advanced Model Targets.

Target Size and Origin

The size of a Model Target is associated with the size of the Bounding Box of the (CAD) Model generated from the MTG Tool. The box size depends on the CAD Model used as input in the MTG.

A Model Target's coordinate system has its origin at the origin of the 3D model used to define the target. This contrasts with other types of Observers available with Vuforia Engine, such as Image Targets (Its origin is the center of the image).

Therefore, you need to know and be aware of the origin of the 3D model to do any AR augmentation in relation to the physical object because you cannot change the origin from the API.

Model Target size and origin are implicitly defined by the 3D model used when generating the Model Target.

Target and Target Pose Unit

Theunits used for Model Targets (and their pose) are in meters.

Create a Model Target Observer

Detect and track objects by creating a Model Target Observer from a Model Target database. A Model Target Observer is configured with a valid databasePath, targetName, and activeGuideViewName. See Model Target Guide View below. By default, the Model Target Observer is activated when created.

1234567891011
Copy
// Create an Observer config VuModelTargetConfig modelTargetConfig = vuModelTargetConfigDefault(); modelTargetConfig.databasePath = "ModelTargetDatabase.xml"; modelTargetConfig.targetName = "ModelTargetName"; // Select a Guide View to be active modelTargetConfig.activeGuideViewName = "GuideView_Front"; // Create a Model Target Observer VuObserver* modelTargetObserver = { NULL }; VuModelTargetCreationError creationError; vuEngineCreateModelTargetObserver(engine, &modelTargetObserver, &modelTargetConfig, &creationError);

Configuring and creating an Observer with non-default additional optional arguments should be done while other Observers from the same database are deactivated to avoid performance constraints.

Observations

Observations are reported in the state and deliver information on a Model Target Observer's pose status and status info. Target information for a Model Target Observer is available in the VuModelTargetObservationTargetInfo struct. See the Observer and Observations article for more details.

Creation of Observation list

123456789
Copy
// Create an observation list VuObservationList* obsList = { NULL }; vuObservationListCreate(&obsList); // Get and parse model target observations list vuStateGetModelTargetObservations(state,obsList); int32_t listSize = 0; vuObservationListGetSize(obsList, &listSize);

Then, parse the observation list to get the target’s information.

1234567891011121314151617
Copy
// Parse the model target list for (int i = 0; i < listSize; i++) { VuObservation* obs = { NULL }; vuObservationListGetElement(obsList, i, &obs); VuPoseInfo poseInfo; vuObservationGetPoseInfo(obs, &poseInfo); VuModelTargetObservationTargetInfo targetInfo; vuModelTargetObservationGetTargetInfo(obs, &targetInfo); if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE) { // Do something with poseInfo and targetInfo } }

Destroy Observer and Observation

Destroy objects and processes after usage to free up memory.

Call the following to destroy the Model Target ObservationList and the Model Target Observer:

Destroy ObservationList

12
Copy
// Destroy the observation list vuObservationListDestroy(obsList);

Destroy the Observer

12
Copy
// Destroy the observer vuObserverDestroy(modelTargetObserver);

Enhanced Runtime Detection for Standard Model Targets

Improve a Standard Model Target’s detection performance by caching local appearance information at runtime. The cached data is updated every time you track the Model Target and saved after the session or when deactivating the Model Target.

This feature mainly enhances detection performance for Standard Targets, though it is also available for Advanced Model Targets. However, the enhanced detection performance effects might not be as prominent for Advanced Model Targets as for standard Model Targets. Also, consider that enabling the feature incurs some additional memory overhead and runtime processing.

NOTE: This feature has no effect on Model Targets that use VU_TRACKING_OPTIMIZATION_LOW_FEATURE_OBJECTS.

Enable enhanced runtime detection

Set the enhanceRuntimeDetection parameter in the VuModelTargetConfig to VU_TRUE to enable the Vuforia Engine to store detection data locally on the device. It only takes effect when the corresponding Model Target has been loaded.

12
Copy
VuModelTargetConfig modelTargetConfig = vuModelTargetConfigDefault(); modelTargetConfig.enhanceRuntimeDetection = VU_TRUE;

When enabled, the cached data is stored in a folder named ModelTargetDetectionCache in the private storage location of the application, with each target having its unique subfolder named with the target’s unique ID. Get a target’s ID using the vuModelTargetObserverGetTargetUniqueId() function.

Cached data

When the Model Target is deactivated, the cached data is updated asynchronously; any immediate call to vuObserverActivate() or vuObserverDestroy() will, therefore, wait until the cache update of the Model Target Observer is finished, potentially causing a delay.

If there is insufficient free space and a cache update fails, a warning message will be logged, and it will be necessary to free up disk space on the device.

Clear all stored cached data for all Model Targets using the vuEngineClearModelTargetObserverDetectionCache() function. If a target is detected again, the caching will automatically restart.

1
Copy
vuEngineClearModelTargetObserverDetectionCache(VuEngine* engine);

Calling this function will fail if there is an active Model Target Observer.

NOTE: Clearing the cache may take a significant amount of time. Therefore, it is recommended that this method not be called on the main UI thread.

Resetting Tracking

When using Runtime Enhanced Detection, it is recommended to include a way for application users to reset tracking in cases where e.g., a strong misalignment is seen and is not corrected automatically. Calling vuModelTargetObserverReset(), which resets the tracking of a Model Target Observer, also removes any existing cached data for a Model Target. If the target is detected again, the caching will automatically restart.

Guide Views

Non-Advanced Model Targets require a Guide View to be displayed on the screen to assist users in finding the correct position from where the object can be successfully detected. If you are unfamiliar with Guide Views, please refer to the Model Target Guide View page for a more detailed introduction to the Guide View concept.

Accessing Guide Views

Querying the list of available guide views:

123
Copy
vuGuideViewListCreate(VuGuideViewList** list); vuModelTargetObserverGetGuideViews(const VuObserver* observer, VuGuideViewList* list); vuGuideViewListGetSize(const VuGuideViewList* list, int32_t* listSize);

Accessing a specific guide view from the list

1
Copy
vuGuideViewListGetElement(const VuGuideViewList* list, int32_t element, VuGuideView** guideView);

Destroy GuideViewList

1
Copy
vuGuideViewListDestroy(VuGuideViewList* list);

Switching between Guide Views

For Model Targets with multiple Guide Views, you can individually select them by calling get/set to the Model Target Observer.

1
Copy
vuModelTargetObserverSetActiveGuideViewName(VuObserver* observer, const char* name);

Advanced Model Targets will automatically recognize objects without requiring a Guide View to be set and rendered. Setting an active Guide View to an Advanced Model Target will return VU_FAILED.

Custom Guide View Rendering and Positioning

In the API, the VuGuideView exposes the following properties of a Guide View:

NOTE: Guide Views use a Y-up convention. See Frame of Reference for more details.

  • Guide View Intrinsic Parameters: The intrinsics used to render the Guide View are represented as a camera calibration data structure and match the intrinsics of the current device's camera.
1
Copy
vuGuideViewGetIntrinsics(const VuGuideView* guideView, VuCameraIntrinsics* cameraIntrinsics);
  • Guide View Extrinsic Parameters: The extrinsics represent the model view matrix used to render the Guide View. It is represented as a Matrix4x4 data structure.
  • Overlay image: A rendering of the object in an abstracted style (edge rendering), as it would appear from the Guide View position in the MTG (i.e., using the initial View Intrinsic Parameters and View Extrinsic Parameters).

If you change the Guide View pose, you can use the MTG to modify the Detection Position and re-export the Model Target database. See How to Create Model Targets for details.

If you want to change the Guide View pose at runtime, you can use the get/set Guide View pose to modify the pose and, consequently, the position from which tracking will begin.

12
Copy
vuGuideViewGetPose(const VuGuideView* guideView, VuMatrix44F* pose); vuGuideViewSetPose(VuGuideView* guideView, const VuMatrix44F* pose);

Guide Views and Advanced Model Targets

Advanced Model Targets do not require Guide Views to detect the object. The Observer configuration’s activeGuideviewName should, therefore, be set to null. However, to assist users in identifying the object of an Advanced Model Target, a Guide View can be created at runtime using vuModelTargetObserverGetGuideViewForAdvanced and displayed during status info VU_MODEL_TARGET_OBSERVATION_STATUS_INFO_INITIALIZING. When this function is called, the Vuforia Engine creates a best estimate Guide View image (VuGuideView) based on the Model Target’s Advanced View range and then renders an outline of the model on the screen. The Guide View for Advanced Model Targets has no effect on detecting the object and only serves as visual guidance. The pose of the Guide View and the rendered image can be updated with vuGuideViewSetPose without affecting the object’s detection capability.

Mesh Observer for Model Targets

The Mesh Observer API provides access to the geometry of the Model Target Observer at runtime. You can use the mesh to render special effects, such as occlusion, custom 3D guide views, or detection animations. See Create Mesh Observer from Model Target for creating Mesh Observers.

Tracking Optimization

Vuforia Engine provides multiple modes that optimize tracking for different scenarios and cases. You can set the mode in the Optimize Tracking tab in the MTG or at runtime through the VuTrackingOptimization API to one of the following values.

  • VU_TRACKING_OPTIMIZATION_DEFAULT
  • VU_TRACKING_OPTIMIZATION_LOW_FEATURE_OBJECTS
  • VU_TRACKING_OPTIMIZATION_AR_CONTROLLER

Use the get/set methods to set the tracking optimization of a Model Target before starting Vuforia Engine.

12345
Copy
// Get a Model Target Observer’s tracking optimization mode vuModelTargetObserverGetTrackingOptimization(VuObserver* observer, VuTrackingOptimization optimization); // Set a Model Target Observer’s tracking optimization mode vuModelTargetObserverSetTrackingOptimization(VuObserver* observer, VuTrackingOptimization optimization);

Setting the tracking optimization mode on a tracked Model Target Observer will reset the tracking of the target.

See more information on tracking optimization at Optimizing Model Target Tracking.

Model Target Pose Status and Status Info

A Model Target’s pose status and status info indicate how well an object is tracked. Possible pose status values are _NO_POSE, _TRACKED, _EXTENDED_TRACKED, or _LIMITED. See Pose Status and Status Info for a table with pose status and status info.

We recommend enabling Extended Tracking when using any type of Model Targets to provide a robust experience, especially with large-scale objects. Extended Tracking is supported through Device Tracking.

The reported tracking status will be _TRACKED when the Model Target tracking is active and robust and _EXTENDED_TRACKED when the Device Pose Observation returns the last observed position while the Model Target is out of camera view or occluded. In situations where the Device Pose Observation provides insufficient information to generate an accurate _EXTENDED_TRACKED pose of the Model Target, the tracking status will be set to _LIMITED.

When _LIMITED tracking status is reported, we recommend hiding augmentations of the Model Target until more robust tracking is reported again.

Status Info – Wrong Scale

Use VuModelTargetObservationGetStatusInfo to get additional information on the tracking status info. A _WRONG_SCALE will be reported if the Model Target is determined to be scaled incorrectly and does not match the size of the physical object. A _WRONG_SCALE is reported in three situations.

  • The Model Target is reported as _TRACKED, but its status info returns _WRONG_SCALE. The target is tracked reliably, but poses will be reported in a non-metrical coordinate system that matches the configured size of the Model Target.
  • The Model Target is reported as _EXTENDED_TRACKED, but its status info returns _WRONG_SCALE. The target is indirectly tracked but poses will be reported in a non-metrical coordinate system that matches the configured size of the Model Target.
  • The Model Target is reported as _LIMITED, and the status info is _WRONG_SCALE. Automatic scale correction is not possible, tracking is limited and with low accuracy.

 

In both situations, Vuforia Engine has detected scale issues to a degree where you should consider examining the parity of the physical and digital models.

If the Model Target’s scale differs significantly from the physical object’s, tracking and augmentations can drift. Resolve scaling issues by regenerating your Model Target with a correct scale. See Best Practices for Scaling Model Targets for additional options.

Standard Model Target Behavior

For standard Model Targets, a Model Target Observation has the following status info collected in the state after activation with

1
Copy
VuModelTargetObservationTargetInfo == VU_OBSERVATION_POSE_STATUS_NO_POSE;

and

1
Copy
VuModelTargetObservationStatusInfo == VU_MODEL_TARGET_OBSERVATION_STATUS_INFO_RECOMMENDING_GUIDANCE;

Tracking starts once the user has aligned the object with the Guide View, and the target’s status info switches to VU_OBSERVATION_POSE_STATUS_TRACKED.

Advanced Model Target Behavior

For Advanced Model Targets with a detection range up to 360°, displaying a Guide View at runtime is generally unnecessary. When an Advanced Model Target database is activated, all objects are initialized in the state with the status info:

1
Copy
VuModelTargetObservationTargetInfo == VU_OBSERVATION_POSE_STATUS_NO_POSE

and

1
Copy
VuDevicePoseObservationStatusInfo == VU_DEVICE_POSE_OBSERVATION_STATUS_INFO_INITIALIZING;

When one of the Model Targets is detected, tracking starts automatically, and its state switches to VU_OBSERVATION_POSE_STATUS_EXTENDED_TRACKED.

Controlling Recognition Behavior for Advanced Model Targets with Multiple Models

As long as an Advanced Model Target is _TRACKED, detecting other Model Targets is automatically disabled to save processing power and prevent unwanted activation of the other targets. When tracking of a Model Target is changed from _TRACKED to _EXTENDED_TRACKED or _LIMITED, recognition is automatically enabled again to allow detection of other activated Model Targets (or to relocate the current Model Target). Since only a single Model Target can be actively tracked at a time, recognition and activation of another Model Target can lead to complete tracking loss of the previously tracked Model Target.

NOTE: _LIMITED pose status with _WRONG_SCALE status info does not re-enable recognition of other Model Targets.

In scenarios where this behavior is not desired (e.g., to save power or when only a single Model Target is used in a session), this behavior can be turned off by calling:

12
Copy
VuBool enable = VU_FALSE; vuEngineSetModelTargetRecoWhileExtendedTracked(VuEngine* engine, VuBool enable);

The default value is then set to VU_TRUE, which can only be set when the Vuforia Engine is not running. When set to VU_FALSE, the first successfully tracked Model Target will stay active until its associated Observer is deactivated.

Configuring Model Targets

In this section, we present additional common configuration options for Model Targets.

Size

Get a Model Target's size, height, length, and width in meters.

1
Copy
vuModelTargetObserverGetTargetSize(const VuObserver* observer, VuVector3F* size);

Scale

Scale the Model Target with a scale factor to one of the retrieved values from the GetTargetSize().

1
Copy
vuModelTargetObserverSetTargetScale(VuObserver* observer, float scale);

Bounding Box

Use this function to get an axis-aligned bounding box of an image from its respective Observer relative to the target’s frame of reference.

1
Copy
vuModelTargetObserverGetAABB(const VuObserver* observer, VuAABB* bbox);

 

Can this page be better?
Share your feedback via our issue tracker