Log in or register to post comments

How To Access the Camera Image in Unity

November 23, 2015 - 9:47am #1

 

I use the script attached in the following link. 

https://developer.vuforia.com/library/articles/Solution/How-To-Access-the-Camera-Image-in-Unity

This post seem very old. Many class , attribute doesn't exist now.

How to access the camera Image (without augmentation)?

 

I tried following : 

Texture2D texture2D = textureGameObject.GetComponent<Renderer>().material.mainTexture as Texture2D ; 

(textureGameObject is the  BackgroundPlane gameObject, the grandson of ARCamera prefab in Vuforia.)

 

I can get this texture easily in Mac OSX , But this get nothing on IOS 9.1.

What is the right way to access the camera Image (without augmentation)?

 

Thanks!!

 

 

How To Access the Camera Image in Unity

February 23, 2018 - 6:43am #6

Hello, bugaboostudio.

How do you have done this part: I stored the pixel results (line 74, pixels[0], pixels[1], pixels[2]) in a Color32 variable?

I used this exemple: http://www.imgportal.net/imgresources/?page_id=287

But i keep getting this error:

Exception in callback: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> UnityEngine.UnityException: LoadRawTextureData: not enough data provided (will result in overread).

  

Best regards, Rafael.

How To Access the Camera Image in Unity

February 22, 2018 - 7:41am #5

I don't know whether it's late or not, but I got it working with this version of your script using vuforia 7. I used RGBA8888 as PIXEL_FORMAT (line 9) and I stored the pixel results (line 74, pixels[0], pixels[1], pixels[2]) in a Color32 variable . I also had to replace VuforiaBehaviour by VuforiaARController in Start method.

I hope this helps someone in trouble.

How To Access the Camera Image in Unity

November 26, 2015 - 6:21am #4

The following script is a variation of the code snippet found in the original article, but using the latest Vuforia SDK 5.0 API:

Also, note that you may want to try different Frame Formats, e.g. one of RGB888 or RGB565:

using UnityEngine;
using System.Collections;
using Vuforia;
public class CameraAccess : MonoBehaviour 
{
    private bool mAccessCameraImage = true;
    
    // The desired camera image pixel format
    private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.GRAYSCALE;// or RGBA8888, RGB888, RGB565, YUV
    // Boolean flag telling whether the pixel format has been registered
    private bool mFormatRegistered = false;
	void Start () 
    {
        // Register Vuforia life-cycle callbacks:
        VuforiaBehaviour.Instance.RegisterVuforiaStartedCallback(OnVuforiaStarted);
        VuforiaBehaviour.Instance.RegisterOnPauseCallback(OnPause);
        VuforiaBehaviour.Instance.RegisterTrackablesUpdatedCallback(OnTrackablesUpdated);
    }
    /// <summary>
    /// Called when Vuforia is started
    /// </summary>
    private void OnVuforiaStarted()
    {
        // Try register camera image format
        if (CameraDevice.Instance.SetFrameFormat(mPixelFormat, true))
        {
            Debug.Log("Successfully registered pixel format " + mPixelFormat.ToString());
            mFormatRegistered = true;
        }
        else
        {
            Debug.LogError("Failed to register pixel format " + mPixelFormat.ToString() + 
                "\n the format may be unsupported by your device;" +
                "\n consider using a different pixel format.");
            mFormatRegistered = false;
        }
    }
    /// <summary>
    /// Called when app is paused / resumed
    /// </summary>
    private void OnPause(bool paused)
    {
        if (paused)
        {
            Debug.Log("App was paused");
            UnregisterFormat();
        }
        else
        {
            Debug.Log("App was resumed");
            RegisterFormat();
        }
    }
    /// <summary>
    /// Called each time the Vuforia state is updated
    /// </summary>
    private void OnTrackablesUpdated()
    {
        if (mFormatRegistered)
        {
            if (mAccessCameraImage)
            {
                Vuforia.Image image = CameraDevice.Instance.GetCameraImage(mPixelFormat);
                if (image != null)
                {
                    string imageInfo = mPixelFormat + " image: \n";
                    imageInfo += " size: " + image.Width + " x " + image.Height + "\n";
                    imageInfo += " bufferSize: " + image.BufferWidth + " x " + image.BufferHeight + "\n";
                    imageInfo += " stride: " + image.Stride;
                    Debug.Log(imageInfo);
                    byte[] pixels = image.Pixels;
                    if (pixels != null && pixels.Length > 0)
                    {
                        Debug.Log("Image pixels: " + pixels[0] + "," + pixels[1] + "," + pixels[2] + ",...");
                    }
                }
            }
        }
    }
    /// <summary>
    /// Unregister the camera pixel format (e.g. call this when app is paused)
    /// </summary>
    private void UnregisterFormat()
    {
        Debug.Log("Unregistering camera pixel format " + mPixelFormat.ToString());
        CameraDevice.Instance.SetFrameFormat(mPixelFormat, false);
        mFormatRegistered = false;
    }
    /// <summary>
    /// Register the camera pixel format
    /// </summary>
    private void RegisterFormat()
    {
        if (CameraDevice.Instance.SetFrameFormat(mPixelFormat, true))
        {
            Debug.Log("Successfully registered camera pixel format " + mPixelFormat.ToString());
            mFormatRegistered = true;
        }
        else
        {
            Debug.LogError("Failed to register camera pixel format " + mPixelFormat.ToString());
            mFormatRegistered = false;
        }
    }
}

 

How To Access the Camera Image in Unity

November 26, 2015 - 1:17am #3

 

using UnityEngine;
using UnityEngine.UI;
using Vuforia;
using System.Collections;
using Image = Vuforia.Image;

public class ShowWebcamTexture : MonoBehaviour
{
    public Text stateText;
    public Material mat;
    private Texture2D texture;
    private int frameCount = 0;
    private Vuforia.Image.PIXEL_FORMAT _curFormat;
    private Vuforia.Image.PIXEL_FORMAT _targetFormat;
    // Use this for initialization
    void Start ()
	{
	    texture = new Texture2D(512, 512);
        mat.mainTexture = texture;
        GetComponent<Renderer>().sharedMaterial.mainTexture = texture;
        _targetFormat = _curFormat = Vuforia.Image.PIXEL_FORMAT.UNKNOWN_FORMAT;
	    stateText.text = "Idle";
        Debug.Log((int)Image.PIXEL_FORMAT.GRAYSCALE);
        Debug.Log((int)Image.PIXEL_FORMAT.RGB888);
        Debug.Log((int)Image.PIXEL_FORMAT.RGBA8888);

	    VuforiaBehaviour.Instance.RegisterTrackablesUpdatedCallback(OnTrackablesUpdated);
	}

    public void OnTrackablesUpdated()
    {
        if (_targetFormat != _curFormat)
        {
            if (CameraDevice.Instance.SetFrameFormat(_targetFormat, true))
            {
                stateText.text = "succeed";
                _curFormat = _targetFormat;
            }
            else
            {
                stateText.text = "failed";
                _targetFormat = _curFormat = Image.PIXEL_FORMAT.UNKNOWN_FORMAT;
            }
        }
        if (_curFormat == Image.PIXEL_FORMAT.UNKNOWN_FORMAT) return;

        var image = CameraDevice.Instance.GetCameraImage(_curFormat);
        if (image == null)
        {
            stateText.text = "error image";
            return;
        }

        image.CopyToTexture(texture);
    }
    
    public void SetImageFormat(int formatInt)
    {
        _targetFormat = (Image.PIXEL_FORMAT)formatInt;
    }
}

 

Same problem. Cant get camera image either.The texture is Dark.

How To Access the Camera Image in Unity

November 23, 2015 - 10:31am #2
I modify the script for Vuforia 5 , then it works.

using UnityEngine;
using System.Collections;
using Vuforia ;  // have to use this 
public class CameraImageAccess : MonoBehaviour
{

	private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
	private bool m_RegisteredFormat = false;
	private bool m_LogInfo = true;
	void Start()
	{
                //qcarBehaviour is rename to VuforiaBehaviour
		VuforiaBehaviour qcarBehaviour = (VuforiaBehaviour) FindObjectOfType(typeof(VuforiaBehaviour));
		if (qcarBehaviour)
		{
			qcarBehaviour.RegisterTrackablesUpdatedCallback(OnTrackablesUpdated);
		}
	}
	public void OnTrackablesUpdated()
	{
		if (!m_RegisteredFormat)
		{
			CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
			m_RegisteredFormat = true;
		}
		if (m_LogInfo)
		{
			CameraDevice cam = CameraDevice.Instance;
			Image image = cam.GetCameraImage(m_PixelFormat);
			if (image == null)
			{
				Debug.Log(m_PixelFormat + " image is not available yet");
			}
			else
			{
				string s = m_PixelFormat + " image: \n";
				s += "  size: " + image.Width + "x" + image.Height + "\n";
				s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
				s += "  stride: " + image.Stride;
				Debug.Log(s);
				m_LogInfo = false;
			}
		}
	}
}

 

Log in or register to post comments