Log in or register to post comments

[Camera] Get the image from the camera

July 17, 2012 - 9:57am #1

Hi everybody, 
this question was already asked, but I've some issues ton get one image from the AR camera.

I'm using this code:

</p><p class="p1">CameraDevice cam = CameraDevice.Instance;</p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1">if</span>(cam == <span class="s2">null</span>)</p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>{</p><p class="p2"><span class="s3"><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span></span><span class="s1">    Debug</span><span class="s3">.Log(</span>"Error cam null !"<span class="s3">);</span></p><p class="p3"><span class="s3"><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span></span>    yield<span class="s3"> </span>return<span class="s3"> </span><span class="s2">null</span><span class="s3">;</span></p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>}</p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>Image image = cam.GetCameraImage(Image.PIXEL_FORMAT.UNKNOWN_FORMAT);</p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s1">if</span>(image != <span class="s2">null</span> && image.IsValid())</p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>{</p><p class="p1"><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span>    temp = <span class="s1">new</span><span class="s1">Texture2D</span>(image.Width, image.Height,<span class="s1">TextureFormat</span>.RGB24, <span class="s2">false</span>);</p><p class="p1"><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span><span class="Apple-tab-span"></span>    temp.LoadImage(image.Pixels);</p><p class="p1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>}</p><p class="p1">

My camera is not null but image is null. I checked with GetAllImage from the camera it returns always a dictonnary with 0 values into it. The camera is displaying correctly on iPad 3, I have no detection problem, or some issues like that.

I tried different format, but if the GetAllImage returns an empty dictonnary, it not a format issue.
 

Thanks ! 

[Camera] Get the image from the camera

March 1, 2015 - 9:09am #17

Dear,
Could you tell me where should I put the code of Image.cs and the code of conversion?

Thanks a lot to helping me.

[Camera] Get the image from the camera

July 30, 2012 - 12:44pm #16

DavidBeard wrote:

@vincekilian how are you calling Application.CaptureScreenShot to capture the video background? - am I reading that right?

 

@DavidBeard:  I was just providing an alternate way of taking a screenshot without GUI elements appearing in the screenshot.  My suggestion takes a screenshot of what's rendered on the screen - not a capture of the video background exclusively.  I should have clarified my response with the subject matter I was referring to in my answer!  My apologies.

 

[Camera] Get the image from the camera

July 30, 2012 - 8:00am #15

Sorry to answer now, I didn't see your question :(

You have both code I added, one for Image.cs (3 answers before) and the one that I used to convert image from AR to Texture2D (5 answers before).

[Camera] Get the image from the camera

July 18, 2012 - 6:05pm #14

@loic_illogika can you post a complete sample of the code that you're using? We've been discussing the provision of an Image utility, and would be interested in testing your solution.

 

@vincekilian how are you calling Application.CaptureScreenShot to capture the video background? - am I reading that right?

[Camera] Get the image from the camera

July 18, 2012 - 2:06pm #13

Yep.. you can also achieve the inverse effect in the else statement, allowing you to show watermarks or other content that you'd only like to appear on the screenshot itself (e.g, enable a guitext with System.DateTime.Now to create a timestamped screenshot.)

[Camera] Get the image from the camera

July 18, 2012 - 2:00pm #12

Nice trick :D

[Camera] Get the image from the camera

July 18, 2012 - 1:21pm #11

loic_illogika wrote:

Oh, I understand, but the screenshot will take also my GUI that is in front of the camera, renderer in another ortho camera...I would like to avoid this effect and get the current Image from the camera.

You can wrap your GUI code with a bool check (e.g., !takingScreenshot), then when you call Application.CaptureScreenshot just set it to true.  Set it back to false when your screenshot is complete.  Usually I do this in a coroutine so I can yield for the screenshot to finish doing its thing.

 

[Camera] Get the image from the camera

July 18, 2012 - 11:36am #10

Yeah it's working for me, with the RGB888 with manual conversion. I've also a RGB565 conversion, which is working almost perfect (weird blue color).

You should add a function into the API to convert the Image to a Texture2D, like I'm doing. My code is free for use :D

[Camera] Get the image from the camera

July 18, 2012 - 10:47am #9

So is that working for you? I am going to inquire with our devs on this question, because we have frequent requests relating to use of the Image data in this way.

[Camera] Get the image from the camera

July 18, 2012 - 10:38am #8

As I said, the problem come from the API, so I changed the CopyPixelsFormUnmanagedBuffer fonction into the Image.cs file. 

Quote:

 // Called by the QCARBehaviour when the unmanaged data buffer has been

    // filled:

    public void CopyPixelsFromUnmanagedBuffer()

    {

        if (mData == null || mUnmanagedData == IntPtr.Zero)

        {

            Debug.LogError("Image: Cannot copy image image data.");

            return;

        }

 

int iFormatMultiplier = 1;

if(mPixelFormat == PIXEL_FORMAT.RGB565)

iFormatMultiplier = 2;

else if(mPixelFormat == PIXEL_FORMAT.RGB888)

iFormatMultiplier = 3;

        Marshal.Copy(mUnmanagedData, mData, 0, mBufferWidth * mBufferHeight * iFormatMultiplier);

        mDataSet = true;

    }

Because I don't know the YUV size (12 or more bytes) I'm not handling it.
You should use it for future version of Unity AR API ;) 

[Camera] Get the image from the camera

July 18, 2012 - 10:26am #7

I changed my code so now it works approximatly, I cas retrieve the first thirds of the image.
The buffer is 921600 length, but after the first 307200 pixels, all bytes are dark.

I'm using RGB888 format to get the image. 

Quote:

Image image = cam.GetCameraImage(Image.PIXEL_FORMAT.RGB888);

 

if(image != null && image.IsValid())

{

pixelFormat = TextureFormat.RGB24;

rawData = image.Pixels;

localTexture = new Texture2D(image.Width , image.Height , pixelFormat, false);

for(int iPos = 0; iPos < localTexture.height; iPos++)

{

for(int jPos = 0; jPos < localTexture.width; jPos++)

{

byte redByte = rawData[(jPos + (iPos * localTexture.width)) * 3];

byte greenByte = rawData[((jPos + (iPos * localTexture.width)) * 3) + 1];

byte blueByte = rawData[((jPos + (iPos * localTexture.width)) * 3) + 2];

Color currentColor = new Color(redByte / 255f, greenByte / 255f, blueByte / 255f);

localTexture.SetPixel(jPos, localTexture.height - 1 - iPos,currentColor);

}

}

 

}

Note that after I apply the Texture2D and I setted before the format with SetFrameFormat.

[Camera] Get the image from the camera

July 18, 2012 - 10:22am #6

I tried to change the format to RGB565 and I have half of the image into the pixel buffer. It seems like the API will not use the format to determine the source image size.

 

[Camera] Get the image from the camera

July 17, 2012 - 2:10pm #5

Oh, I understand, but the screenshot will take also my GUI that is in front of the camera, renderer in another ortho camera...I would like to avoid this effect and get the current Image from the camera.

I use one script to convert from the image to a texture2D, using RGB888 and RGB24 on Texture2D.

Quote:

for(int iPos = 0; iPos < localTexture.width; iPos++)

{

for(int jPos = 0; jPos < localTexture.height; jPos++)

{

byte redByte = rawData[(jPos + (iPos * localTexture.height) * 3)];

byte greenByte = rawData[(jPos + (iPos * localTexture.height) * 3) + 1];

byte blueByte = rawData[(jPos + (iPos * localTexture.height) * 3) + 2];

Color currentColor = new Color(redByte / 255f, greenByte / 255f, blueByte / 255f);

localTexture.SetPixel(iPos, jPos,currentColor);

}

}

And it's not working ^^

Edit: localTexture is my Texture2D, and rawData is image.Pixels.

[Camera] Get the image from the camera

July 17, 2012 - 10:53am #4

So is the Image rendering properly to the Texture2D?

[Camera] Get the image from the camera

July 17, 2012 - 10:53am #3

So you only want the camera image, not the augmentation as well? Take a look at the Background Texture Access sample, because that's what is demonstrated. The readme file that's included describes two approaches to rendering the camera image to a 2D texture, one of which uses a shader.  But obtaining the pixel array from Image, as you've attempted, is going to give you a raw byte array, which will need to map to Color structs for use by Unity Textures.

If you want a screenshot, you can use Application.CaptureScreenShot(), or..

private Texture2D m_Texture;

void Start()
{
m_Texture = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, true);
}

void OnGUI()
{
if (GUI.Button(new Rect(20, 20, 150, 100), "ReadPixels"))
{
m_Texture.ReadPixels(new Rect(0, 0, Screen.width, Screen.height), 0, 0, true);
m_Texture.Apply();
}
}

[Camera] Get the image from the camera

July 17, 2012 - 10:41am #2

Ok,

it seems that if I wait 1sec after the camera is initialized, then Set the format (SetFrameFormat) wait again and then get the camera image it works :)

Log in or register to post comments