Log in or register to post comments

Just want to get the camera texture...

March 22, 2012 - 11:56am #1

Hey all, first time here.

I was disappointed that the Webcam API does not work on all Androids, esp my Droid4. I need to get the current camera image (live image, not gallery). I don't need any markers, any tracking, no fancy effects, just the plain ole image.
I tried modifying the BackgroundTexturesAccess project, but the best I could get was a rect which does NOT fill the screen (about 60%) and looks like it's one pixel of the image, or mega-blur of it, because it's a solid color of something in front of the lens.

Any help here? It must be pretty simple and minimal, but I've been trying different combos and options all day.

Thanks

Just want to get the camera texture...

July 30, 2012 - 7:57am #18

You should take a look to this thread (from myself):

https://ar.qualcomm.at/content/camera-get-image-camera

Just want to get the camera texture...

July 30, 2012 - 7:07am #17

Hi,

We got a bit stuck by this bug as well - as I am sure you have figured out by now - the bug is that pixel encoding is not considered in: CopyPixelsFromUnmanagedBuffer

 

Cheers

Re: Just want to get the camera texture...

March 26, 2012 - 10:34pm #16

Thanks for the report, we'll look into this. Do note that most often the pixel buffer returned is the camera image inside a power-of-two buffer. The BufferWidth and BufferHeight properties of the Image tell you the size of the buffer, while the Width and Height properties tell you the size of the Image inside that buffer.

Could you point out the bug you found? Were you able to fix it for your purposes?

Thanks,

- Kim

Re: Just want to get the camera texture...

March 26, 2012 - 2:01pm #15

Found bug in Image class.

Re: Just want to get the camera texture...

March 26, 2012 - 1:21pm #14

Correction, it's 1/2. I get blackness for the first 320 lines, then good image thereafter. Flipping on Y, that means I get the top 1/2 ok. The rest is black. This is the result of CameraDevice.Instance.GetCameraImage(Image.PIXEL_FORMAT_RGB888).Pixels

Re: Just want to get the camera texture...

March 26, 2012 - 12:42pm #13

Ok, I'm finally able to get something, but it's strange.
The imageSize.x,y = 640,480, sounds ok. I need to transfer this to a Color32[], so I scan those into a Color32 array that's 640x480 in length.
1) It's upside down, so I need to copy in Y-reverse order, but the PROBLEM:
2) It's a partial - only about 1/3 of the image shows up, the rest is black. I get no errors about array out of bounds or any thing like that.

Now, I do create my Texture2D by using 640x480, is that wrong? Should it be padded out to next-largest power of 2? I don't see that requirement in the Unity docs, and I would think that if it were so, I'd have obvious horizontal sync problems, but I don't. The image looks normal, just short.

Re: Just want to get the camera texture...

March 22, 2012 - 5:50pm #12

RGB888, yes.

Re: Just want to get the camera texture...

March 22, 2012 - 5:49pm #11

Thanks for the code. I just got done analyzing the pixels it returns and figured out the bit-order for 565. But if I can set it to RGB24, that will save me some coding!

Re: Just want to get the camera texture...

March 22, 2012 - 5:35pm #10

[QUOTE" />Any idea what format byte[" /> pixels is in?[/QUOTE" />

That's what you're establishing when calling SetFrameFormat.

which..
[QUOTE" />
Enables the delivery of certain pixel formats via the State object.

Per default the state object will only contain images in formats that are required for internal processing, such as gray scale for tracking. setFrameFormat() can be used to enforce the creation of images with certain pixel formats. Notice that this might include additional overhead.[/QUOTE" />

So Kim's class registers Image.PIXEL_FORMAT.RGB565, and then retrieves these Images via

[CODE" />Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);[/CODE" />

where

[CODE" />private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;[/CODE" />

Re: Just want to get the camera texture...

March 22, 2012 - 4:58pm #9

I would use RT's (would be easy) but I'd like it to work for Indie.

Re: Just want to get the camera texture...

March 22, 2012 - 4:57pm #8

This looks promising. Any idea what format byte[] pixels is in? RGB, RGBA, floats in a byte array, 565, etc?

Re: Just want to get the camera texture...

March 22, 2012 - 4:09pm #7
ksiva wrote:

If you're just interested in the raw pixels you can use the CameraDevice class to get an Image object. Use the SetFrameFormat method to register for images of a certain format (see the Image class for details).

I suggest implementing the ITrackerEventHandler interface and using the OnTrackablesUpdated callback to ensure you get the latest camera image each frame.

public class MyImageHandler : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;

    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }

        CameraDevice.Instance.SetFrameFormat(mPixelFormat, true);
    }

    public void OnTrackablesUpdated()
    {
        Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);

        if (cameraImage != null)
        {
            byte[] pixels = cameraImage.Pixels;
            // Do something with the pixels!
        }
    }
}

- Kim

Thank you, I searched for it the whole day.

Also you can use RenderTexture and then copy to Texture2D and getting the color array if you dont care about the speed.

Re: Just want to get the camera texture...

March 22, 2012 - 4:01pm #6

If you're just interested in the raw pixels you can use the CameraDevice class to get an Image object. Use the SetFrameFormat method to register for images of a certain format (see the Image class for details).

I suggest implementing the ITrackerEventHandler interface and using the OnTrackablesUpdated callback to ensure you get the latest camera image each frame.

public class MyImageHandler : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;

    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }

        CameraDevice.Instance.SetFrameFormat(mPixelFormat, true);
    }

    public void OnTrackablesUpdated()
    {
        Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);

        if (cameraImage != null)
        {
            byte[] pixels = cameraImage.Pixels;
            // Do something with the pixels!
        }
    }
}

- Kim

Re: Just want to get the camera texture...

March 22, 2012 - 3:42pm #5

I get the background image object, then (not working with QCAR, does work with webcam):

var tx : Texture2D = background.renderer.material.mainTexture;
var pix = tx.GetPixels32();

I'm pulling features out of the camera image for real-time texturing, let's say. I'm not too concerned about the speed of it, it only happens infrequently and at user's request.

Re: Just want to get the camera texture...

March 22, 2012 - 3:34pm #4

@DaveA can you post a code snippet showing how you're obtaining the frame? What are trying to do with it?

Re: Just want to get the camera texture...

March 22, 2012 - 3:17pm #3

Is there is a way to get the pixels buffer (maybe from java) even in raw mode?
Thats a better aproach, GetPixels is very slow

Re: Just want to get the camera texture...

March 22, 2012 - 3:14pm #2

Ok I got the BackgroundTexturesAccess to show my camera ok, but I still can't access pixels (like with GetPixels32() ). I need to get those. How?

Log in or register to post comments