The Session Recorder allows you to record data from system devices (e.g., camera and sensors) and tracking data (e.g., the device position or pose). You can play back the recorded data using the File Driver on a device to recreate a Vuforia session offline. In some instances, it may also be useful to share the recording with the Vuforia team to aid in reproducing a reported issue.
Prerequisites
Supported cameras
Devices with cameras that capture frames in the following pixel formats are supported:
- NV21
- NV12
- YUV420P
- YV12
NOTE: It is required that the device contains a hardware-accelerated H.264 video encoder which supports encoding images in the pixel format delivered by the camera.
Record sessions in one view orientation
Record a session only in one view (display) orientation (portrait or landscape) and make sure to set the orientation before starting the Vuforia Engine. If the view orientation is not known, vuRecordingStart
will not start a recording and return error code VU_RECORDING_START_ERROR_UNKNOWN_ORIENTATION. If the VuViewOrientation
is changed during an ongoing recording, the recording will be stopped, and the status info will be set to VU_RECORDING_STATUS_INFO_ERROR_ORIENTATION_CHANGED. See Rendering in Native for more information on setting the orientation.
Make a Session Recording
There are two main classes for recording sessions: VuRecording
that manages the recording session itself, and the VuSessionRecorderController
that lets you create, and destroy Session Recording instances.
NOTE: VuSessionRecorderController
also provides a means to remove all previous Session Recordings. This is useful for instance if the device is running low on available space.
Create a recording
Create a new VuRecording
instance by configuring the sources to record data from. Setting start to VU_TRUE will start the recording immediately.
12345678CopyVuRecordingConfig config = vuRecordingConfigDefault();
config.start = VU_TRUE;
vuSessionRecorderControllerGetDefaultRecordingDataFlags(recorderController, &config.dataFlags);
VuRecordingCreationError creationError {};
VuRecording* recording {};
vuSessionRecorderControllerCreateRecording(recorderController, &config, &recording, &creationError);
Use the optional parameter config.outputDirectory
to specify a custom path for storing the recording.
Optionally, also set the encoding quality. This can be useful for reducing the storage requirements of longer recordings. Note that recordings made with the VU_RECORDING_VIDEO_ENCODING_QUALITY_MEDIUM setting may not be suitable for playback with AR tracking enabled.
Additional recording configuration options
The recording frame rate (config.frameRate
) is by default set to VU_RECORDING_FRAME_RATE_AUTO which lets Vuforia Engine choose the frame rate best fitting the performance of the device. Other frame rate options are:
- VU_RECORDING_FRAME_RATE_FULL – record frames at the full frame rate of the camera.
- VU_RECORDING_FRAME_RATE_HALF – record frames at half the frame rate of the camera.
The camera recording resolution (config.scale
) is likewise set to VU_RECORDING_IMAGE_SCALE_AUTO by default to let Vuforia Engine choose the best image scale in accordance with the device’s performance capabilities. Other image scale options are:
- VU_RECORDING_IMAGE_SCALE_FULL – record images at full resolution.
- VU_RECORDING_IMAGE_SCALE_HALF – resample images in half resolution along both dimensions.
The recording quality (config.videoEncodingQuality
) is set to VU_RECORDING_VIDEO_ENCODING_QUALITY_HIGH by default but can be set to a lower setting if longer Session Recordings captured on less powerful devices are needed. The options are:
- VU_RECORDING_VIDEO_ENCODING_QUALITY_HIGH – record with minimal compression artifacts.
VU_RECORDING_VIDEO_ENCODING_QUALITY_MEDIUM – record for general purpose recordings with video encoding allowing for longer captures without resulting in too large a file. The recording may not be suitable for playback with AR tracking.
Destroying a recording instance
When a recording is completed, the VuRecording
instance can be cleaned up using vuRecordingDestroy(recording, VU_TRUE)
. The second argument determines if the recorded data should be deleted from storage as well. If it is set to VU_FALSE, only the instance in memory will be destroyed.
To destroy all currently existing VuRecording
instances, call
1Copy
vuSessionRecorderControllerDestroyRecordings(recorderController, VU_TRUE);
Again, the second argument determines if the recorded data should be deleted as well.
Permissions
In order to support audio recording capabilities in the Session Recorder Controller, your app must fulfill audio-related, platform-specific permission and configuration requirements. Your app is responsible for executing the following platform-specific actions prior to invoking vuEngineCreate()
. See also Engine Permissions for other necessary permissions checks to initialize Vuforia Engine.
Android |
Add the |
iOS |
Microphone usage description: Add the key Microphone access: Request access to the microphone by calling the |
UWP |
Add the following snippet to the 123
|
Lumin |
Add the |
If the aforementioned platform-specific permission and configuration requirements are not fulfilled by the app, then:
- The
vuSessionRecorderControllerGetSupportedRecordingDataFlags
call won't report audio support (i.e.,VU_RECORDING_DATA_AUDIO_BIT will be missing). - The
vuSessionRecorderControllerCreateRecording
call will fail with VU_RECORDING_START_ERROR_DATA_SOURCE ifVuRecordingConfig.dataFlags
is configured to record audio.
Querying available sources
The VuRecordingDataFlags
bitmask provides a means to configure the sources to record from. The vuSessionRecorderControllerGetSupportedRecordingDataFlags
method can be used to query the available and supported sources on the current device.
Use vuSessionRecorderControllerGetDefaultRecordingDataFlags
to get the sources that are recommended for recording on the current device. Using the recommended set of sources will always result in a recording that can be used for playback with AR tracking.
Operations on a Recording
After a VuRecording
instance has been created, not only can it be used for making the actual Session Recording, but it also allows for querying various Session Recording properties, as well as for managing the Session Recording lifecycle.
Starting and stopping a recording
A recording can be started and stopped respectively by the vuRecordingStart
and vuRecordingStop
methods of VuRecording
. Start returns VU_SUCCESS if the recording has been successfully started. Starting a recording requires that a Vuforia Engine is running (including the camera). Calling the vuRecordingStart
twice, i.e. a subsequent call after a successful start, has no effect and will fail with the error code ERROR_ANOTHER_RECORDING_RUNNING.
Querying the location of a recording
After successfully starting a recording, vuRecordingGetPath
can be called to retrieve the path to the file where the recording is stored, also if you specified an output directory in the configuration.
Pausing an application
VuSessionRecorderController
does not provide a pause/resume capability. If a recording is running, it is automatically stopped when vuEngineStop
is called. This also means that a recording will be stopped if the application is paused. On resume, vuRecordingStart
should again be called explicitly to start a new recording.
Querying the recording status
The recording status can be queried during a recording using the vuRecordingGetStatus
method and additional information can be retrieved using vuRecordingGetStatusInfo
. This will tell the caller if the recording has for instance been aborted due to the device running out of space (Note that VuRecording
will abort an ongoing recording when the free space available on the device falls below a pre-defined threshold, i.e., before the device runs out of space. In that case, the error will be indicated by the VU_RECORDING_STATUS_INFO_ERROR_INSUFFICIENT_FREE_SPACE info status value).
Removing previous recordings
The SessionRecorder API provides a vuSessionRecorderControllerCleanRecordedData
method for removing all recorded data. This is convenient for instance if the recorded data are not needed any more, and space needs to be freed on the device storage. Please note that cleaning recorded data will work only when no recording is in progress.
Output Format
Each Session Recording is saved to an MP4 file in the private storage space of the app. The file name has the form VuforiaSession-[DATE]-[TIME].mp4, where [DATE] has the form YYYYMMDD (year, month, day) and [TIME] has the form HHMMSS (hour, minute, second). The MP4 file contains the recorded video and the captured tracking data (sensors, poses, etc.) in separate tracks.
Most generic video playback applications can be used to view the recorded video. However, it is not possible to access the recorded tracking data using these applications.
NOTE: If the MP4 file is modified by any third-party tool (e.g., video editor, converter, or similar) it will become unusable for offline playback with Vuforia.