Session Recorder Native API

The Session Recorder allows you to record data from system devices (e.g., camera and sensors) and tracking data (e.g., the device position or pose). You can play back the recorded data using the File Driver on a device to recreate a Vuforia session offline. In some instances, it may also be useful to share the recording with the Vuforia team to aid in reproducing a reported issue.

Prerequisites

Supported cameras

Devices with cameras that capture frames in the following pixel formats are supported:

  • NV21
  • NV12
  • YUV420P
  • YV12

NOTE: It is required that the device contains a hardware-accelerated H.264 video encoder which supports encoding images in the pixel format delivered by the camera.

Record sessions in one view orientation

Record a session only in one view (display) orientation (portrait or landscape) and make sure to set the orientation before starting the Vuforia Engine. If the view orientation is not known, vuRecordingStart will not start a recording and return error code VU_RECORDING_START_ERROR_UNKNOWN_ORIENTATION. If the VuViewOrientation is changed during an ongoing recording, the recording will be stopped, and the status info will be set to VU_RECORDING_STATUS_INFO_ERROR_ORIENTATION_CHANGED. See Rendering in Native for more information on setting the orientation.

Make a Session Recording

There are two main classes for recording sessions: VuRecording that manages the recording session itself, and the VuSessionRecorderController that lets you create, and destroy Session Recording instances.

NOTE: VuSessionRecorderController also provides a means to remove all previous Session Recordings. This is useful for instance if the device is running low on available space.

Create a recording

Create a new VuRecording instance by configuring the sources to record data from. Setting start to VU_TRUE will start the recording immediately.

12345678
Copy
VuRecordingConfig config = vuRecordingConfigDefault(); config.start = VU_TRUE; vuSessionRecorderControllerGetDefaultRecordingDataFlags(recorderController, &config.dataFlags); VuRecordingCreationError creationError {}; VuRecording* recording {}; vuSessionRecorderControllerCreateRecording(recorderController, &config, &recording, &creationError);

Use the optional parameter config.outputDirectory to specify a custom path for storing the recording.

Optionally, also set the encoding quality. This can be useful for reducing the storage requirements of longer recordings. Note that recordings made with the VU_RECORDING_VIDEO_ENCODING_QUALITY_MEDIUM setting may not be suitable for playback with AR tracking enabled.

Additional recording configuration options

The recording frame rate (config.frameRate) is by default set to VU_RECORDING_FRAME_RATE_AUTO which lets Vuforia Engine choose the frame rate best fitting the performance of the device. Other frame rate options are:

  • VU_RECORDING_FRAME_RATE_FULL – record frames at the full frame rate of the camera.
  • VU_RECORDING_FRAME_RATE_HALF – record frames at half the frame rate of the camera.

The camera recording resolution (config.scale) is likewise set to VU_RECORDING_IMAGE_SCALE_AUTO by default to let Vuforia Engine choose the best image scale in accordance with the device’s performance capabilities. Other image scale options are:

  • VU_RECORDING_IMAGE_SCALE_FULL – record images at full resolution.
  • VU_RECORDING_IMAGE_SCALE_HALF – resample images in half resolution along both dimensions.

The recording quality (config.videoEncodingQuality) is set to VU_RECORDING_VIDEO_ENCODING_QUALITY_HIGH by default but can be set to a lower setting if longer Session Recordings captured on less powerful devices are needed. The options are:

  • VU_RECORDING_VIDEO_ENCODING_QUALITY_HIGH – record with minimal compression artifacts.

VU_RECORDING_VIDEO_ENCODING_QUALITY_MEDIUM – record for general purpose recordings with video encoding allowing for longer captures without resulting in too large a file. The recording may not be suitable for playback with AR tracking.

Destroying a recording instance

When a recording is completed, the VuRecording instance can be cleaned up using vuRecordingDestroy(recording, VU_TRUE). The second argument determines if the recorded data should be deleted from storage as well. If it is set to VU_FALSE, only the instance in memory will be destroyed.

To destroy all currently existing VuRecording instances, call

1
Copy
vuSessionRecorderControllerDestroyRecordings(recorderController, VU_TRUE);

Again, the second argument determines if the recorded data should be deleted as well.

Permissions

In order to support audio recording capabilities in the Session Recorder Controller, your app must fulfill audio-related, platform-specific permission and configuration requirements. Your app is responsible for executing the following platform-specific actions prior to invoking vuEngineCreate(). See also Engine Permissions for other necessary permissions checks to initialize Vuforia Engine.

Android

Add the "android.permission.RECORD_AUDIO" permission to the AndroidManifest.xml and request it at runtime in your app's code

iOS

Microphone usage description: Add the key "NSMicrophoneUsageDescription" to the Info.plist with an explanation why your app requires microphone access, otherwise the app will be rejected by the App Store submission process.

Microphone access: Request access to the microphone by calling the "AVCaptureDevice requestAccessForMediaType" API with the media type "AVMediaTypeAudio" and request it at runtime in your app’s code

UWP

Add the following snippet to the <Capabilities> tag in your app's Package.appxmanifest to enable audio recording capabilities:

123
Copy
<Capabilities> <DeviceCapability Name="microphone" /> </Capabilities>

Lumin

Add the "android.permission.RECORD_AUDIO" permission to the AndroidManifest.xml and request it at runtime in your app's code

If the aforementioned platform-specific permission and configuration requirements are not fulfilled by the app, then:

  • The vuSessionRecorderControllerGetSupportedRecordingDataFlags call won't report audio support (i.e.,VU_RECORDING_DATA_AUDIO_BIT will be missing).
  • The vuSessionRecorderControllerCreateRecording call will fail with VU_RECORDING_START_ERROR_DATA_SOURCE if VuRecordingConfig.dataFlags is configured to record audio.

Querying available sources

The VuRecordingDataFlags bitmask provides a means to configure the sources to record from. The vuSessionRecorderControllerGetSupportedRecordingDataFlags method can be used to query the available and supported sources on the current device.

Use vuSessionRecorderControllerGetDefaultRecordingDataFlags to get the sources that are recommended for recording on the current device. Using the recommended set of sources will always result in a recording that can be used for playback with AR tracking.

Operations on a Recording

After a VuRecording instance has been created, not only can it be used for making the actual Session Recording, but it also allows for querying various Session Recording properties, as well as for managing the Session Recording lifecycle.

Starting and stopping a recording

A recording can be started and stopped respectively by the vuRecordingStart and vuRecordingStop methods of VuRecording. Start returns VU_SUCCESS if the recording has been successfully started. Starting a recording requires that a Vuforia Engine is running (including the camera). Calling the vuRecordingStart twice, i.e. a subsequent call after a successful start, has no effect and will fail with the error code ERROR_ANOTHER_RECORDING_RUNNING.

Querying the location of a recording

After successfully starting a recording, vuRecordingGetPath can be called to retrieve the path to the file where the recording is stored, also if you specified an output directory in the configuration. 

Pausing an application

VuSessionRecorderController does not provide a pause/resume capability. If a recording is running, it is automatically stopped when vuEngineStop is called. This also means that a recording will be stopped if the application is paused. On resume, vuRecordingStart should again be called explicitly to start a new recording.

Querying the recording status

The recording status can be queried during a recording using the vuRecordingGetStatus method and additional information can be retrieved using vuRecordingGetStatusInfo. This will tell the caller if the recording has for instance been aborted due to the device running out of space (Note that VuRecording will abort an ongoing recording when the free space available on the device falls below a pre-defined threshold, i.e., before the device runs out of space. In that case, the error will be indicated by the VU_RECORDING_STATUS_INFO_ERROR_INSUFFICIENT_FREE_SPACE info status value).

Removing previous recordings

The SessionRecorder API provides a vuSessionRecorderControllerCleanRecordedData method for removing all recorded data. This is convenient for instance if the recorded data are not needed any more, and space needs to be freed on the device storage. Please note that cleaning recorded data will work only when no recording is in progress.

Output Format

Each Session Recording is saved to an MP4 file in the private storage space of the app. The file name has the form VuforiaSession-[DATE]-[TIME].mp4, where [DATE] has the form YYYYMMDD (year, month, day) and [TIME] has the form HHMMSS (hour, minute, second). The MP4 file contains the recorded video and the captured tracking data (sensors, poses, etc.) in separate tracks.
Most generic video playback applications can be used to view the recorded video. However, it is not possible to access the recorded tracking data using these applications.

NOTE: If the MP4 file is modified by any third-party tool (e.g., video editor, converter, or similar) it will become unusable for offline playback with Vuforia.

Can this page be better?
Share your feedback via our issue tracker