Dear,
Could you tell me where should I put the code of Image.cs and the code of conversion?

Thanks a lot to helping me.

DavidBeard wrote:

@vincekilian how are you calling Application.CaptureScreenShot to capture the video background? - am I reading that right?

@DavidBeard:  I was just providing an alternate way of taking a screenshot without GUI elements appearing in the screenshot.  My suggestion takes a screenshot of what's rendered on the screen - not a capture of the video background exclusively.  I should have clarified my response with the subject matter I was referring to in my answer!  My apologies.

Sorry to answer now, I didn't see your question :(

You have both code I added, one for Image.cs (3 answers before) and the one that I used to convert image from AR to Texture2D (5 answers before).

@loic_illogika can you post a complete sample of the code that you're using? We've been discussing the provision of an Image utility, and would be interested in testing your solution.

@vincekilian how are you calling Application.CaptureScreenShot to capture the video background? - am I reading that right?

Yep.. you can also achieve the inverse effect in the else statement, allowing you to show watermarks or other content that you'd only like to appear on the screenshot itself (e.g, enable a guitext with System.DateTime.Now to create a timestamped screenshot.)

Nice trick :D

loic_illogika wrote:

Oh, I understand, but the screenshot will take also my GUI that is in front of the camera, renderer in another ortho camera...I would like to avoid this effect and get the current Image from the camera.

You can wrap your GUI code with a bool check (e.g., !takingScreenshot), then when you call Application.CaptureScreenshot just set it to true.  Set it back to false when your screenshot is complete.  Usually I do this in a coroutine so I can yield for the screenshot to finish doing its thing.

Yeah it's working for me, with the RGB888 with manual conversion. I've also a RGB565 conversion, which is working almost perfect (weird blue color).

You should add a function into the API to convert the Image to a Texture2D, like I'm doing. My code is free for use :D

So is that working for you? I am going to inquire with our devs on this question, because we have frequent requests relating to use of the Image data in this way.

As I said, the problem come from the API, so I changed the CopyPixelsFormUnmanagedBuffer fonction into the Image.cs file. 

Quote:

 // Called by the QCARBehaviour when the unmanaged data buffer has been

    // filled:

    public void CopyPixelsFromUnmanagedBuffer()

    {

        if (mData == null || mUnmanagedData == IntPtr.Zero)

        {

            Debug.LogError("Image: Cannot copy image image data.");

            return;

        }

 

int iFormatMultiplier = 1;

if(mPixelFormat == PIXEL_FORMAT.RGB565)

iFormatMultiplier = 2;

else if(mPixelFormat == PIXEL_FORMAT.RGB888)

iFormatMultiplier = 3;

        Marshal.Copy(mUnmanagedData, mData, 0, mBufferWidth * mBufferHeight * iFormatMultiplier);

        mDataSet = true;

    }

Because I don't know the YUV size (12 or more bytes) I'm not handling it.
You should use it for future version of Unity AR API ;) 

I changed my code so now it works approximatly, I cas retrieve the first thirds of the image.
The buffer is 921600 length, but after the first 307200 pixels, all bytes are dark.

I'm using RGB888 format to get the image. 

Quote:

Image image = cam.GetCameraImage(Image.PIXEL_FORMAT.RGB888);

 

if(image != null && image.IsValid())

{

pixelFormat = TextureFormat.RGB24;

rawData = image.Pixels;

localTexture = new Texture2D(image.Width , image.Height , pixelFormat, false);

for(int iPos = 0; iPos < localTexture.height; iPos++)

{

for(int jPos = 0; jPos < localTexture.width; jPos++)

{

byte redByte = rawData[(jPos + (iPos * localTexture.width)) * 3];

byte greenByte = rawData[((jPos + (iPos * localTexture.width)) * 3) + 1];

byte blueByte = rawData[((jPos + (iPos * localTexture.width)) * 3) + 2];

Color currentColor = new Color(redByte / 255f, greenByte / 255f, blueByte / 255f);

localTexture.SetPixel(jPos, localTexture.height - 1 - iPos,currentColor);

}

}

 

}

Note that after I apply the Texture2D and I setted before the format with SetFrameFormat.

I tried to change the format to RGB565 and I have half of the image into the pixel buffer. It seems like the API will not use the format to determine the source image size.

Oh, I understand, but the screenshot will take also my GUI that is in front of the camera, renderer in another ortho camera...I would like to avoid this effect and get the current Image from the camera.

I use one script to convert from the image to a texture2D, using RGB888 and RGB24 on Texture2D.

Quote:

for(int iPos = 0; iPos < localTexture.width; iPos++)

{

for(int jPos = 0; jPos < localTexture.height; jPos++)

{

byte redByte = rawData[(jPos + (iPos * localTexture.height) * 3)];

byte greenByte = rawData[(jPos + (iPos * localTexture.height) * 3) + 1];

byte blueByte = rawData[(jPos + (iPos * localTexture.height) * 3) + 2];

Color currentColor = new Color(redByte / 255f, greenByte / 255f, blueByte / 255f);

localTexture.SetPixel(iPos, jPos,currentColor);

}

}

And it's not working ^^

Edit: localTexture is my Texture2D, and rawData is image.Pixels.

So is the Image rendering properly to the Texture2D?

So you only want the camera image, not the augmentation as well? Take a look at the Background Texture Access sample, because that's what is demonstrated. The readme file that's included describes two approaches to rendering the camera image to a 2D texture, one of which uses a shader.  But obtaining the pixel array from Image, as you've attempted, is going to give you a raw byte array, which will need to map to Color structs for use by Unity Textures.

If you want a screenshot, you can use Application.CaptureScreenShot(), or..

private Texture2D m_Texture;

void Start()
{
m_Texture = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, true);
}

void OnGUI()
{
if (GUI.Button(new Rect(20, 20, 150, 100), "ReadPixels"))
{
m_Texture.ReadPixels(new Rect(0, 0, Screen.width, Screen.height), 0, 0, true);
m_Texture.Apply();
}
}

Ok,

it seems that if I wait 1sec after the camera is initialized, then Set the format (SetFrameFormat) wait again and then get the camera image it works :)