Calibrate your camera device and provide custom camera intrinsics to your Vuforia Driver. Camera calibration improves Vuforia Engine’s performance of detection and tracking.
Camera calibration is the mechanism for estimating the parameters of the lens and the image sensor of a camera. These parameters are known as intrinsics and are used by Vuforia Engine to correct lens distortion.
Prerequisites
To calibrate a camera, and get the camera intrinsics, you will need a camera calibration tool. There are various tools available for use; or you have the option to make your own calibration program. See OpenCV: Camera Calibration.
Depending on your solution, your steps will include:
- A printed calibration template file - preferably with stiff backing.
- The camera and a way to capture camera frames from it.
- A process to calibrate based on the camera frames.
- An updated custom driver with the calibration results. Alternatively, you can also sideload the intrinsics. See Using External Camera with UWP.
Calibration Guidelines
This section provides general guidelines on how to obtain calibration data and applying it with a custom driver.
Calibrating an external camera for the Vuforia Engine Driver Framework requires obtaining its camera intrinsics from calibration software. The intrinsics that Vuforia Engine needs the parameters are frame size, focal length, and principal point, all provided in pixels, and that the distortion values should be reported in the same order as the OpenCV lens distortion model.
- Frame size, 2 integer numbers for width and height in pixels.
- Focal length, 2 floating point numbers for x and y distance between focal point and the image plane in pixels.
- Principal point, 2 floating point numbers for x and y center of the image in pixels.
- Distortion values, 2-8 floating point numbers representing radial and tangential distortion in camera frame. Distortion coefficients are in the following form (r: radial, t: tangential): [r0, r1, t0, t1, r2, r3, r4, r5]. See the distortion models in the camera JSON configuration table.
For more information, please refer to the Provide the Camera Intrinsics section below.
Print calibration template
A calibration template must be printed in order to perform camera calibration. Please refer to the specific calibration tool‘s instructions for information on which template to use.
Printing considerations:
- Template size should be at least A4.
- Print the template on a rigid foam board or mount the printout on a flat, rigid surface such as a sheet of glass, foamboard, an aluminum sheet or a plexiglass sheet. Take care not to introduce kinks, ripples or distortions.
Capture images
To calibrate the camera, capture images of the template file using the camera's supplied software. Use compressed formats like JPG, but ensure images are sharp and have minimal compression artifacts.
Examples of capture software:
Capturing Calibration Frames
- Specify the camera frame size in the application used to capture the images. Different frame sizes will have different camera parameters; therefore, the specified frame size used in the calibration process must be the same frame size that will be used by the Driver in Vuforia Engine.
- Capture images of the printed calibration template file from different distances and angles as shown below. For best results during calibration, ensure the following when capturing the images:
- The template fills into the corners of the image in some views.
- Depending on the calibration tool, the template may need to be fully visible in every image, including a white border.
- The calibration area is well lit and without spectral reflections from the environment or the template target. A light level of 600 lux is recommended.
- The template corners are distributed to all positions in the image frame especially the corners and sides of the image frame.
- Capture at least 20 images from different angles and distances. Capturing upwards of 30 images can produce better calibration results.
Example of capturing images from different angles:
Process Camera Frames
Once calibration frames are captured, the frames must be processed in the tool of choice.
After completing the camera calibration, the frame size, focal length, principal point and distortion parameters are reported. These values are used from within the Driver.
Provide the Camera Intrinsics
Once the calibration data has been obtained, the Driver needs to be updated to provide the camera intrinsics to Vuforia Engine through the CameraIntrinsics
structure. The CameraIntrinsics
structure is a value in the CameraFrame
structure which is provided on every frame through the CameraCallBack::onNewCameraFrame(frame)
function.
For an improved performance, deliver the intrinsics on a per-frame basis as cameras with auto-focus can change the focal length. Similarly, Optical Image Stabilization (OIS) may also change the principal point between frames.
For more information on building a custom driver, please refer to Creating a Camera Driver.
CameraIntrinsics
Structure
focalLengthX
– Horizontal Focal Length result from the calibration procedure.focalLengthY
– Vertical Focal Length result from the calibration procedure.principalPointX
– Horizontal Principal Point result from the calibration procedure.principalPointY
– Vertical Principal Point result from the calibration procedure.distortionCoefficients
– Values in the Distortion result from the calibration procedure from left-to-right order.
TIP: The focal length (fx, fy), principal point (cx, cy), and distortion model should not be zero; the principal point should be a point somewhat close to the center of the image.
NOTE: If the Camera Calibration tool reports values with +/- error values, you can ignore them.
NOTE: Vuforia Engine supports the OpenCV lens distortion models with up to 8 parameters. In the case the selected distortion model has less than 8 parameters, the remaining ones should be filled with zeros.
For more details on the mathematical details behind camera calibration, the intrinsics, and distortion models, see the OpenCV Camera Calibration Documentation.