You should take a look to this thread (from myself):

https://ar.qualcomm.at/content/camera-get-image-camera

Hi,

We got a bit stuck by this bug as well - as I am sure you have figured out by now - the bug is that pixel encoding is not considered in: CopyPixelsFromUnmanagedBuffer

Cheers

Thanks for the report, we'll look into this. Do note that most often the pixel buffer returned is the camera image inside a power-of-two buffer. The BufferWidth and BufferHeight properties of the Image tell you the size of the buffer, while the Width and Height properties tell you the size of the Image inside that buffer.

Could you point out the bug you found? Were you able to fix it for your purposes?

Thanks,

- Kim

Found bug in Image class.

Correction, it's 1/2. I get blackness for the first 320 lines, then good image thereafter. Flipping on Y, that means I get the top 1/2 ok. The rest is black. This is the result of CameraDevice.Instance.GetCameraImage(Image.PIXEL_FORMAT_RGB888).Pixels

Ok, I'm finally able to get something, but it's strange.
The imageSize.x,y = 640,480, sounds ok. I need to transfer this to a Color32[], so I scan those into a Color32 array that's 640x480 in length.
1) It's upside down, so I need to copy in Y-reverse order, but the PROBLEM:
2) It's a partial - only about 1/3 of the image shows up, the rest is black. I get no errors about array out of bounds or any thing like that.

Now, I do create my Texture2D by using 640x480, is that wrong? Should it be padded out to next-largest power of 2? I don't see that requirement in the Unity docs, and I would think that if it were so, I'd have obvious horizontal sync problems, but I don't. The image looks normal, just short.

RGB888, yes.

Thanks for the code. I just got done analyzing the pixels it returns and figured out the bit-order for 565. But if I can set it to RGB24, that will save me some coding!

[QUOTE" />Any idea what format byte[" /> pixels is in?[/QUOTE" />

That's what you're establishing when calling SetFrameFormat.

which..
[QUOTE" />
Enables the delivery of certain pixel formats via the State object.

Per default the state object will only contain images in formats that are required for internal processing, such as gray scale for tracking. setFrameFormat() can be used to enforce the creation of images with certain pixel formats. Notice that this might include additional overhead.[/QUOTE" />

So Kim's class registers Image.PIXEL_FORMAT.RGB565, and then retrieves these Images via

[CODE" />Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);[/CODE" />

where

[CODE" />private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;[/CODE" />

I would use RT's (would be easy) but I'd like it to work for Indie.

This looks promising. Any idea what format byte[] pixels is in? RGB, RGBA, floats in a byte array, 565, etc?

ksiva wrote:

If you're just interested in the raw pixels you can use the CameraDevice class to get an Image object. Use the SetFrameFormat method to register for images of a certain format (see the Image class for details).

I suggest implementing the ITrackerEventHandler interface and using the OnTrackablesUpdated callback to ensure you get the latest camera image each frame.

public class MyImageHandler : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;

    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }

        CameraDevice.Instance.SetFrameFormat(mPixelFormat, true);
    }

    public void OnTrackablesUpdated()
    {
        Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);

        if (cameraImage != null)
        {
            byte[] pixels = cameraImage.Pixels;
            // Do something with the pixels!
        }
    }
}

- Kim

Thank you, I searched for it the whole day.

Also you can use RenderTexture and then copy to Texture2D and getting the color array if you dont care about the speed.

If you're just interested in the raw pixels you can use the CameraDevice class to get an Image object. Use the SetFrameFormat method to register for images of a certain format (see the Image class for details).

I suggest implementing the ITrackerEventHandler interface and using the OnTrackablesUpdated callback to ensure you get the latest camera image each frame.

public class MyImageHandler : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;

    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }

        CameraDevice.Instance.SetFrameFormat(mPixelFormat, true);
    }

    public void OnTrackablesUpdated()
    {
        Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);

        if (cameraImage != null)
        {
            byte[] pixels = cameraImage.Pixels;
            // Do something with the pixels!
        }
    }
}

- Kim

I get the background image object, then (not working with QCAR, does work with webcam):

var tx : Texture2D = background.renderer.material.mainTexture;
var pix = tx.GetPixels32();

I'm pulling features out of the camera image for real-time texturing, let's say. I'm not too concerned about the speed of it, it only happens infrequently and at user's request.

@DaveA can you post a code snippet showing how you're obtaining the frame? What are trying to do with it?

Is there is a way to get the pixels buffer (maybe from java) even in raw mode?
Thats a better aproach, GetPixels is very slow

Ok I got the BackgroundTexturesAccess to show my camera ok, but I still can't access pixels (like with GetPixels32() ). I need to get those. How?