Vuforia Engine Camera API Overview¶

The Vuforia Engine Camera API provides access to a device's camera and enables control of the camera parameters. Use them to optimize camera functionality for certain platforms and changing environments.

The camera is tied to the Vuforia Engine lifecycle and is initialized and deinitialized with the Engine. To learn more about the camera and how it is used in your AR session, please see Working with the Camera.

Camera Configuration¶

Adjust the camera parameters with the CameraController, which includes setting the video mode before starting Vuforia Engine and setting the exposure mode and focus mode after Vuforia Engine has started.

CameraController¶

Video mode¶

When developing apps for certain devices, it is sensible to adjust the video mode in accordance with the device's performance capabilities. With the camera video modes, you can optimize for quality but with more strain on the computational load. Or you can optimize for speed to minimize the impact Vuforia Engine puts on the device.

Call get/set to get the active mode and configure it to a preferred mode.

vuCameraControllerGetActiveVideoMode(const VuController* controller, VuCameraVideoModePreset* cameraVideoModePreset);
vuCameraControllerSetActiveVideoMode(VuController* controller, VuCameraVideoModePreset cameraVideoModePreset);

Enumerator for `VuCameraVideoModePreset`
VU_CAMERA_VIDEO_MODE_PRESET_DEFAULT	Best compromise between speed and quality.
VU_CAMERA_VIDEO_MODE_PRESET_OPTIMIZE_SPEED	Minimize Vuforia Engine impact on the system.
VU_CAMERA_VIDEO_MODE_PRESET_OPTIMIZE_QUALITY	Optimize for Vuforia Engine performance. Application performance could go down.

VU_CAMERA_VIDEO_MODE_PRESET_DEFAULT mode is used if nothing else is set.

Change the mode to VU_CAMERA_VIDEO_MODE_PRESET_OPTIMIZE_SPEED if the Vuforia Engine performance in this mode meets your needs. If you expect the target to be moved continuously, we recommend using the default mode.

Focus mode¶

Control the camera focus mode to continuously autofocus the camera or choose to let the user focus the camera to a fixed point through a user interaction. Please note that some devices may not support all the focus modes available.

Check which focus modes are supported on a device:

vuCameraControllerIsFocusModeSupported(VuController* controller, VuCameraFocusMode focusMode, VuBool* isFocusModeSupported);

Set the focus mode at runtime:

vuCameraControllerGetFocusMode(const VuController* controller, VuCameraFocusMode* focusMode);
vuCameraControllerSetFocusMode(VuController* controller, VuCameraFocusMode focusMode);

Focus mode enum `VuCameraFocusMode`	Behavior
VU_CAMERA_FOCUS_MODE_FIXED	Sets the camera into a fixed focus defined by the camera driver.
VU_CAMERA_FOCUS_MODE_TRIGGERAUTO	Triggers a single autofocus operation. After the operation is completed, the focus mode will automatically change to VU_CAMERA_FOCUS_MODE_FIXED. Setting a `CameraRegionOfInterest` requires VU_CAMERA_FOCUS_MODE_TRIGGERAUTO to be set afterwards.
VU_CAMERA_FOCUS_MODE_CONTINUOUSAUTO	Lets you turn on driver-level continuous autofocus for cameras. This mode is recommended as it guarantees that the camera is always focused on the target.
VU_CAMERA_FOCUS_MODE_INFINITY	Sets the camera to infinity, as provided by the camera driver implementation. This mode will set the camera to always focus on the background elements in the scene. (Only supported on UWP and Android without ARCore).
VU_CAMERA_FOCUS_MODE_MACRO	Sets the camera to macro mode, as provided by the camera driver implementation. This mode provides a sharp camera image for closeups of approximately 15 cm, rarely used in AR setups. (Not supported on iOS and Magic Leap).

If nothing else is set, the platform's default focus mode is used.

Focus region¶

Warning

NOTE: The camera controls for setting/getting focus and exposure regions are not supported on devices running ARCore. Android devices with ARCore disabled can use the controls.

In some situations, you may need to focus on a particular region of the camera frame. The Camera Controller allows this by setting a focus region with a region-of-interest center point and extent as a percentage of the camera frame's width and height.

NOTE: Adjusting the focus to more extreme values might negatively affect detection and tracking of Vuforia targets and can result in blurry images.

First check if the device platform supports setting the focus region. Call this while Vuforia Engine is running.

vuCameraControllerIsFocusRegionSupported(const VuController* controller,
VuBool* isFocusRegionSupported );  

Set the focus region using vuCameraRegionOfInterest:

VuCameraRegionOfInterest focusROI;
focusROI.center = { 0.2f, 0.2f };
focusROI.extent = 0.5f;
vuCameraControllerSetFocusRegion(VuController* controller, VuCameraRegionOfInterest focusROI);

Reset the focus region setting by setting the vuCameraRegionOfInterest back to center = 0.5f, 0.5f, and extent = 1.0f.

NOTE: The focus extent percentage is ignored on iOS devices but must still be set to a value.

vuCameraControllerSetFocusRegion() must be followed by setting focus mode to VU_CAMERA_FOCUS_MODE_TRIGGERAUTO to trigger the re-focus. After the re-focus, the mode sets itself to VU_CAMERA_FOCUS_MODE_FIXED.

The focus region is not retained across Vuforia Engine start and stop. For example, the focus region will revert to the default if an app is paused and then resumed.

Exposure mode¶

Control the camera exposure to improve the image quality in changing lighting settings. Set the mode to continuous autoexposure, or let the user trigger a single auto-exposure operation or set the exposure as fixed. Please note that some devices may not support all the exposure modes available.

Check if an exposure mode is supported on a device:

vuCameraControllerIsExposureModeSupported(VuController* controller, VuCameraExposureMode exposureMode, VuBool* isExposureModeSupported);

If nothing else is set, the platform's default exposure mode is used.

Exposure mode `VuCameraExposureMode`
VU_CAMERA_EXPOSURE_MODE_TRIGGERAUTO	Triggers a single auto-exposure operation. After the operation is completed, the exposure mode will automatically change to VU_CAMERA_EXPOSURE_MODE_FIXED. Setting a `CameraRegionOfInterest` requires VU_CAMERA_EXPOSURE_MODE_TRIGGERAUTO to be set afterwards.
VU_CAMERA_EXPOSURE_MODE_CONTINUOUSAUTO	Allows the device to control auto-exposure continuously. This mode is recommended as it guarantees that the camera is always applying exposure corrections on the camera view.
VU_CAMERA_EXPOSURE_MODE_FIXED	Sets the exposure at a fixed value defined by the camera driver.

In most scenarios, the VU_CAMERA_EXPOSURE_MODE_CONTINUOUSAUTO is the recommended mode. Apply other modes when the environment has either strong light or not enough to determine the target.

Setting or changing the exposure mode, while the Vuforia Engine is running, might take a little longer on certain devices.

Exposure region¶

Warning

NOTE: The camera controls for setting/getting focus and exposure regions are not supported on devices running ARCore. Android devices with ARCore disabled can use the controls.

In other situations, you may need to alter the exposure of the camera frames in a specific region of your viewport. The Camera Controller allows this by setting an exposure region with a region-of-interest center point and extent as a percentage of the camera frame's width and height.

NOTE: Adjusting the exposure to more extreme values might negatively affect detection and tracking of Vuforia targets and can result in under- or overexposed images.

First, check if the device platform supports setting the exposure region. Call this while Vuforia Engine is running.

vuCameraControllerIsExposureRegionSupported(const VuController* controller, VuBool* isExposureRegionSupported);

Set the exposure region using vuCameraRegionOfInterest:

VuCameraRegionOfInterest exposureROI;
exposureROI.center = { 0.5f, 0.5f };
exposureROI.extent = 1.0f;
vuCameraControllerSetExposureRegion(VuController* controller, VuCameraRegionOfInterest exposureROI);

Reset the exposure region by setting the vuCameraRegionOfInterest back to center = 0.5f, 0.5f, and extent = 1.0f.

NOTE: The exposure extent percentage is ignored on iOS devices but must still be given a value.

vuCameraControllerSetExposureRegion() must be followed by setting exposure mode to VU_CAMERA_EXPOSURE_MODE_TRIGGERAUTO to trigger the re-exposure. After the re-exposure, the mode sets itself to VU_CAMERA_EXPOSURE_MODE_FIXED.

The exposure region is not retained across Vuforia Engine start and stop. For example, the exposure region will revert to the default if an app is paused and then resumed.

Flash Torch¶

Enable or disable Flash in environments with changing or very low illuminance. This can greatly improve detection and tracking. Please note that not all devices have Flash hardware, in which case setting the Flash call will fail.

Warning

NOTE: ARCore must be set to 1.45 or later in your build Gradle file to use Flash on ARCore-enabled Android devices. We recommend only doing so if the Flash is essential to the use case.

Query if Flash is supported.

vuCameraControllerIsFlashModeSupported(VuController* controller, VuBool* isFlashModeSupported);

Enable/disable flash mode if the device supports it:

vuCameraControllerGetFlashMode(const VuController* controller, VuBool* flashMode);
vuCameraControllerSetFlashMode(VuController* controller, VuBool flashMode);

Get Camera Frames from the State¶

Firstly, check if the camera provides frames to the state. If vuStateHasCameraFrame returns VU_TRUE, then you can get the camera frame from the state (needed for hitTesting for example) and other useful information such as timestamp, indexing, and list of images.

vuStateHasCameraFrame(const VuState *state);
vuStateGetCameraFrame (const VuState *state, VuCameraFrame **cameraFrame);

Learn More¶

Working with the Camera