Unity - How can I access the camera image?

August 10, 2012 - 3:25pm #1
The Image class

There are two options for obtaining the camera image (without augmentation) from Unity. One is using the Image class. This works much like the native version. First, register for the desired image format using the CameraDevice.SetFrameFormat method:

// Running on device - RGB888
CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.RGB888, true);

// Running in Unity PlayMode - GRAYSCALE
CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.GRAYSCALE, true);

Call this method after QCARBehaviour has had a chance to run its Start method (use the Unity script ordering feature, or do this once in an Update callback). Then, you can retrieve the image using the CameraDevice.GetCameraImage method. You can do this from the ITrackerEventHandler.OnTrackablesUpdated callback to ensure that you retrieve the latest camera image that matches the current frame. Always check to make sure the camera image is not null, as it can take a few frames after registering for an image format for the image to become available. Here is the full script:

using UnityEngine;
using System.Collections;

public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
    private bool m_RegisteredFormat = false;
    private bool m_LogInfo = true;

    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }
    }

    public void OnTrackablesUpdated()
    {
        if (!m_RegisteredFormat)
        {
            CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
            m_RegisteredFormat = true;
        }

        if (m_LogInfo)
        {
            CameraDevice cam = CameraDevice.Instance;
            Image image = cam.GetCameraImage(m_PixelFormat);

            if (image == null)
            {
                Debug.Log(m_PixelFormat + " image is not available yet");
            }
            else
            {
                string s = m_PixelFormat + " image: \n";
                s += "  size: " + image.Width + "x" + image.Height + "\n";
                s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
                s += "  stride: " + image.Stride;
                Debug.Log(s);

                m_LogInfo = false;
            }
        }
    }
}
Background Texture Access

The Image class provides the camera pixels as a byte array. This is useful for some image processing tasks, but sometimes it is preferable to obtain the image as an OpenGL texture. You can do this using the approach demonstrated in the BackgroundTextureAccess sample. Here, instead of letting Vuforia render the camera image natively each frame, you register a Texture2D object to be filled with the camera pixels each frame. See the Readme.txt included with that sample (in the Assets folder) for two ways of rendering the camera image using this approach.

Log in or register to post comments