Generate State-Based Model Targets

This guide shows how to generate a State-Based Model Target with the Model Target Generator (MTG), Web API, and CLI.

Create State-Based Model Targets from Illustration files prepared in Creo Illustrate or from process plans with nodes depicting assembly or disassembly instructions. See Illustration Files and Process Plans for details.

See Model Target Generator for installation instructions.

Content:

Model Target Generator

When generating a State-Based Model Target, the state names will be automatically generated from either the Steps in the Figure (Creo Illustrate) or the name of the corresponding node in the scene hierarchy (Process Plan).

Vuforia Engine does not support special characters. Therefore, we apply changes to replace the unsupported characters and limit the state names to 64 characters. Be aware of those changes to state names in case you refer to them at runtime using the Vuforia Engine API:

In particular, this means:

  • All spaces ' ' will be replaced with underscores '_'
  • All unsupported ASCII characters (allowed: [a-z]|[A-Z]|[0-9]|_|-) will be replaced by a dash '-'
  • All non-ASCII characters will be removed.
  • Names will be truncated to 64 characters.
  • If the name is empty, a default name of 'state' will be used.

Import your 3D model

Follow these steps to create a State-Based Model Target from an existing Creo Illustrate procedure (exported as a separate PVZ) or from a Process Plan (any format):

  1. Open the MTG, click Create, and select Model Target.
  2. In the window, select the model file from your directory. The Interpret this model as a state based model target checkbox is automatically enabled for PVZ files that contain a Creo Illustrate figure.

    NOTE: This cannot be detected automatically if the model is a process plan. In this case, check the box manually.

  3. If the PVZ file has multiple figures (currently not supported), select one from the Select an animation dropdown menu.
  4. The model is loaded with all its states, interpreted from the steps and parts, and shown in an expandable panel on the left. The first state is an added state working as the model's base.

    Double-click on a state in the panel to switch the target's 3D representation to that state. This can be used to review that all states have been imported correctly or to customize views for specific states.

  5. Prepare the Model Target by working through the tabs like any other Model Target generation.
    TIP: Simplification also works with State-Based Model Targets. All hierarchies and parts are retained.

Assign Advanced Views to the states

Assign Advanced Views to all or specific states in the MTG. In many cases, we recommend assigning one Advanced View to all states. For example, an Observation view assigned to all states can be useful for consistently recognizing a target in all its states.

In other cases, only some states require a particular view. For instance, the internals of an assembly are only visible if a cover is removed, and a Constrained Angle Range and Target Extent view that looks at these internals from a close distance could be used.

With this in mind, create your Advanced Views:

  1. Select Create Advanced View and Click Next Step.
  2. Choose the Advanced View that best fits your object and use case, and click Next Step. We can select Full 360 because the demo object used in this example is expected to be rotated and viewed upside down during assembly.
  3. Select between Apply to All states or Apply to selected state.

    NOTE: To generate a State-Based Model Target, at least two states with a Guide View or Advanced View are required. We recommend applying one Advanced View (usually 360° Dome, Full 360°, or a Constrained User Position View) to all states and adding additional Advanced Views to individual states as required.

Add additional Advanced Views

  1. Double-click on a state to add more views.
  2. Click Add View.
  3. Select one of the Advanced Views. Most likely, this would be either the Constrained Angle Range and Target Extent or the Constrained User Positions and Target Extent, which both allow for close-up or restricted views.
  4. Select Apply to selected state and click Create Custom View.
  5. You can double-click a state in the list to see which Advanced Views it is associated with. The associated views are highlighted in green.
  6. Double-click on a state, then double-click on an Advanced View to associate or disassociate that view for that state.
    Additional Views can be added to each state to ensure the target can be recognized in that state from this view.
  7. Click Generate Advanced Model Target and submit the database for training.

Guide Views for Standard State-Based Model Targets

We recommend using Advanced Views over standard Guide Views for State-Based Model Targets. Using Advanced Views ensures that the target's state is recognized and changed automatically during tracking.

If you use standard Guide Views, you must manually set the active Guide View and the active state before the target can be detected from this view in that state (or a similar one). Therefore, set up at least one Guide View to track the object at the start of the experience. Additional Guide Views can be made, for example, to introduce a different entry point from close-up positions of a part that needs to be serviced or assembled. The outline image of a standard Guide View will automatically change at runtime based on the currently set state.

Model Target Web API and Model Target CLI

State-Based Model Targets can be programmatically generated using the Model Target CLI or the Model Target Web API, please see the linked documentation for general instructions.

State Configuration Format

To generate State-Based Model Targets with the MTG CLI or the MTG Web API, a configuration file in JSON format must be provided that lists each state and describes the modifications made to individual parts for each state. Creo Illustrate Illustrations and process plans currently are not supported directly.

The following code snippet shows an example configuration with three states:

1234567891011121314151617181920212223242526272829303132333435363738394041
Copy
{ "version": "1.0", "default_state": "full_object", "states": { "full_object": { "base_scene": 0 }, "object_with_part_removed": { "base_scene": 0, "modified_nodes": [ { "ref": "REMOVABLE_PART.PRT", "visible": false } ] }, "object_with_parts_moved": { "base_scene": 0, "modified_nodes": [ { "ref": "MOVEABLE_PART.PRT", "transform": { "matrix": [ -0.99975000, -0.00679829, 0.0213218, 0.0, 0.00167596, 0.92732500, 0.3742540, 0.0, -0.02231650, 0.37419600, -0.9270810, 0.0, -0.01155430, 0.19471100, -0.4782970, 1.0 ] } }, { "ref": "ANOTHER_MOVEABLE_PART.PRT", "transform": { "translation": [ 0.52363, -0.23453, 0.01325 ], "rotation": [ 0.707107, 0.0, 0.0, 0.707107 ] } } ] } } }

Default State

One state must be referenced by name in the default_state property to describe the initial state of the Model Target.

Base Scene

For each state, the base scene in the input model must be specified. If the input model does not define any scenes, the base_scene property can be set to 0.

Modifying a Part

A part modification in the modified_nodes object consists of the following properties:

Reference

The ref property references the name of the part that is modified.

Visible (optional)

This control sets the visibility of a part and its children. If a node's visible attribute is set to false, a child's visibility can still be enabled by creating an explicit modification of the part with visible set to true.

Transform (optional)

A node's local space transformation can be overridden by supplying a matrix property or any of translation, rotation, and scale properties (also known as TRS properties). Those properties follow the definitions in the GLTF 2.0 standard.

If the matrix property is defined, the TRS properties are disallowed.

Providing the State Configuration

To pass the state configuration file to the CLI or Web API, the file content must be included as a string in the stateBasedConfigurationJsonString attribute in the target generation request. Note that this generally requires escaping of special characters to produce a valid JSON request!

When generating an Advanced State-Based Model Target using the Web API, it is possible to provide a list of valid states for each Advanced View. When omitted, the view is assumed to be visible from all states.

The following snippet shows an abbreviated generation request for an Advanced State-Based Model Target:

1234567891011121314151617
Copy
{ "name": "dataset-name", "targetSdk": "10.22", "models": [{ "name": "model-name", "cadDataBlob": "cad-model-file.glb", "upVector": [0, 1, 0], "stateBasedConfigurationJsonString" : "{\"version\": \"1.0\", \"default_state\": \"full_object\", \"states\": { ... } }", "views": [{ "name": "viewpoint_0000", "recognitionRangesPreset": "OBSERVATION", "targetExtentPreset": "FULL_MODEL", "states" : [ "full_object" ] }] }] }

Test your State-Based Model Target in the Creator App

Test your State-Based Model Target in the Vuforia Creator app and verify that your object is recognized and tracked robustly. Import the database as you would with any other Model Target Database.

  1. An Advanced State-Based Model Target trained with Advanced Views will automatically switch between recognized states and visualize them by changing the outline accordingly.
  2. A standard State-Based Model Target with Guide Views will have arrows on each side of the screen to switch between the states. The states can be switched in the order in which they have been defined and imported into the MTG.

Depending on the currently recognized or manually set state, the outline rendering of the target will be updated accordingly. Keep in mind that Advanced State-Based Model Targets will not always pick the exact right state, just one that closely matches what the camera is currently seeing. This can also mean that a quite different state may be selected, particularly if the distinguishing parts are currently not in view. 

Can this page be better?
Share your feedback via our issue tracker