Log in or register to post comments

Obtain unity camera image (without augmentation)

July 30, 2013 - 11:25pm #1

Hi, I am using Unity 4.2 + Vuforia Unity Extension 2.5.8 to develop an android/iOS AR app.

On of the requirements of the app is to take a screenshot of the camera image with and without AR augmentation.

My problem is that I am unable to obtain the camera image without AR augmentation.

 

I made a script based on the "Image Class" solution of the following dev guide, but the screenshot I obtain is the camera image WITH AR augmentation.

What is exptected was the image WITHOUT agmentation.

https://developer.vuforia.com/resources/dev-guide/unity-camera-image-access

 

#Here is my script. It is attached to the ARCamera object of the Unity scene.

 

using UnityEngine;

using System.Collections;

using System.IO;
 
public class TakePicture : MonoBehaviour {
static bool hideButton = false;
static bool hasTakenShot = false;
 
 
void OnGUI () {
if( !TakePicture.hideButton ){
if(GUI.Button(new Rect(40,80,160,40), "Take Picture")){
TakePicture.hideButton = true;
TakePicture.hasTakenShot = true;
}
}
else{
if(TakePicture.hasTakenShot){
// Take Picture without Augmentation
Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB888;
 
CameraDevice.Instance.SetFrameFormat(mPixelFormat, true);
CameraDevice cam = CameraDevice.Instance;
Image image = cam.GetCameraImage(mPixelFormat);
 
string s = mPixelFormat + " image: \n";
                s += "  size: " + image.Width + "x" + image.Height + "\n";
                s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
                s += "  stride: " + image.Stride;
                Debug.Log(s);
 
Texture2D tex = new Texture2D (image.Width, image.Height, TextureFormat.RGB24, false);
image.CopyToTexture(tex);
tex.ReadPixels (new Rect(0, 0, image.Width, image.Height), 0, 0);
tex.Apply ();
 
byte[] bytes = tex.EncodeToPNG();
Destroy(tex);
 
// For testing purposes, also write to a file in the project folder
File.WriteAllBytes(Application.persistentDataPath + "/SavedScreen.png", bytes);
 
TakePicture.hasTakenShot = false;
}
}
}
}
 
 
I would really appreciate any help on this.
 
Also I do not want to use the "Background Texture Access" of the above guide, since the resolution of the image obtained thought that solution in very low (much lower than the phone screen resolution).
 
Thank you in advance.
 
Joce

Obtain unity camera image (without augmentation)

July 30, 2016 - 2:44am #23

mfahadminhas wrote:

Hi,

Is there any separate namespace for QCARBehaviour?
Because even if I use the same code it gives the error on this line, that the type or namespace "QCARBehaviour" could be found.

I have include Vuforia at the top as well.

Please consult the screenshot attached.

I found the solution, it is deprecated in Vuforia 5.
You have to use VuforiaBehaviour except QCARBehaviour.

Obtain unity camera image (without augmentation)

July 30, 2016 - 2:29am #22

Hi,

Is there any separate namespace for QCARBehaviour?
Because even if I use the same code it gives the error on this line, that the type or namespace "QCARBehaviour" could be found.

I have include Vuforia at the top as well.

Please consult the screenshot attached.

AttachmentSize
Image icon qcar.JPG82.97 KB

Obtain unity camera image (without augmentation)

April 15, 2015 - 7:00am #21

This time I got this error, that's wierd even when I added the false argument ! 

 

Obtain unity camera image (without augmentation)

April 15, 2015 - 2:51am #20

Dears,

I'm using a script for getting image without augmentation using Vuforia 3 and it woks fine.

I moved to Vuforia 4 and I got error attached bellow. (ITrackerEventHandler the type or namespace name could not be found are you missing using directive)

Does any one got the same issue? 

Thank you.  

Obtain unity camera image (without augmentation)

February 23, 2015 - 9:49am #19

Oui je l'ai deja trouver merci : )

Obtain unity camera image (without augmentation)

February 23, 2015 - 8:23am #18

Look at the link post below it should solve your issue, it has the same issue. I found it by searching the forum.

https://developer.vuforia.com/forum/qcar-api/error-no-overload-method-registertrackereventhandler-take

Obtain unity camera image (without augmentation)

February 21, 2015 - 12:31am #17

Thank you for your reply.

I got new error, I put here a capture screen to see. 

Sorry about my questions I'm still beginner with Vuforia.

AttachmentSize
Image icon no overload.jpg229.66 KB

Obtain unity camera image (without augmentation)

February 20, 2015 - 9:58am #16

Add this to your code above void Start()

public void OnInitialized()

    {

    }

Also try grayscale, in the past color capture did not work.

m_PixelFormat = Image.PIXEL_FORMAT.GRAYSCALE

 

Or look at my post below, its basicaly a screen capture thur unity. I make a new camera that sees only my ar object(s) and the Vuforia AR camera sees only the background(cull nothing).

I parent my camera to the vufora AR camera so the postions match.

I placed the code (see my old post below) on the AR camera and it captures a color still before any objects are seen on the screen. This method is slow and will not capture at 30fps. but it returns a color texture thur myTexture. I have it on a timer  to capture every 4 sec but that can be chaged easly.

Obtain unity camera image (without augmentation)

February 20, 2015 - 8:12am #15

Hello, any help !

I would like to get Image access without Augmentationbut I get this error message! please any help! thank you.

 

 

Obtain unity camera image (without augmentation)

August 26, 2014 - 1:04pm #14

I was also only able to get the GRAY video to work so i made this after searching and trying diffrent procedures.

The unity Cameras draw in depth order and then the final frame is show, you need to get video BG from the ar camera before your object are drawn by placing them over the AR camera with a seprate camera that only see your AR .

This code will start to capture the image after 4 seconds(nextcapture) and then every 0.4 seconds(CaptureInt). Your capture code needs to execute in OnPostRender().

1. Create a camera that only see your AR objects using layers.
Clear flags  = depth only
Culling Mask  = "YOUR AR LAYER"
set the postion  = 0,100,0
set rotation = 0,0,0
set scale  = 1,1,1
Set the Depth = 1

2. Set the Vuforia AR camera
Clear flags  = Solid color
culling mask  =  nothing
set the postion = 0,100,0
set rotation = 0,0,0
set scale =  1,1,1
Set the Depth  = -1

3. parent your camera to ARCamera, this way the postion and rotations match.

Place this code on the vuforia ARcamera

    private float CaptureInt = 0.4f;
    private float NextCapture = 4;
    private Texture2D MYtexture;

    public int X = 0;
    public int Y = 0;

    public int MyWidth = 100;
    public int MyHeight = 100;

    public Camera YourArCamera;
    public Camera VuforiaARcamera

    //this will preview the captured texture on this object - for testing
    public GameObject TestPlane;
 

private void OnPostRender(){

       //very important that the camera views match
        YourArCamera.projectionMatrix = VuforiaARcamera.projectionMatrix;

         //you can replace this "if"with your own variable as a triger for capture
        if (Time.time > NextCapture){
            NextCapture = Time.time + CaptureInt;  //set the next capture time
            try {

               //Capture an area 100x100 pixles starting at 0,0 you can place your own cordinates here, make sure that the width and height does not execed the screen size!
                MYtexture = new Texture2D(MyWidthMyHeight);
                MYtexture.ReadPixels(new Rect(XYMyWidthMyHeight),0,0);
                MYtexture.Apply();

               //show a preview for testing
                TestPlane.renderer.material.mainTexture = MYtexture;

               //implement your own routine to save "MYtexture" or manipulate it;

            }catch (Exception e){
                Debug.Log("fail get texture:" + e);
            }
        }//had a memeory leak so dump unsused memeory;
        Resources.UnloadUnusedAssets();
}

 

hope this helps....

 

Obtain unity camera image (without augmentation)

June 5, 2014 - 7:07am #13

Same here, it Works only with GRAYSCALE mode....

Any other way (with RenderTexture ) to get Camera Image Access in Unity.?

Obtain unity camera image (without augmentation)

May 12, 2014 - 1:44am #12

when run this code: CameraDevice.Instance.SetFrameFormat(m_PixelFormattrue);

I get an error said that:"Failed to set frame format";

m_PixelFormat = Image.PIXEL_FORMAT.RGB888;

It seems that only m_PixelFormat = Image.PIXEL_FORMAT.GRAYSCALE  is ok .

 

Obtain unity camera image (without augmentation)

August 7, 2013 - 8:09am #11

Is there a way to get a better resolution, considering that I am already MODE_OPTIMIZE_QUALITY QCAR_Behavior.

No - that is the only way.

N

Obtain unity camera image (without augmentation)

August 7, 2013 - 5:53am #10

One more question, about the resolution of the image without augmentation.

I tried on very high-end devices with high resolution cameras (Galasxy S4) ,

and the maximum resolution i get for the picture without augmentation is 720x480,

which is probably much lower than what the device camera actually captures.

 

Is there a way to get a better resolution, considering that I am already MODE_OPTIMIZE_QUALITY QCAR_Behavior.

 

Thanks,

 

Joce

Obtain unity camera image (without augmentation)

August 7, 2013 - 5:33am #9

Oops... Sorry about that one. It works perfectly.

Thank you again.

 

Joce

Obtain unity camera image (without augmentation)

August 7, 2013 - 5:22am #8

Blue screen is normally when you have forgotten to add your scene to the build settings, so Unity has nothing to work with - hence blue screen of nothing.

N

Obtain unity camera image (without augmentation)

August 7, 2013 - 5:16am #7
Thank you for the clarification about the image flipping. I will search how to achieve it.
 
Sorry to ask again and again, but I really would like to get to script to run since it is the recommended way.
I added a void OnInitialized() function, and tried to run on my android devices.
After Unity splash scree, I get a blue screen and nothing happens.
 
Is this related to Vurforia's version?  (I am using Vuforia 2.5.8)
 
Joce
 
 
using UnityEngine;
using System.Collections;
using System.IO;
 
public class TakePicture : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
    private bool m_RegisteredFormat = false;
    private bool m_LogInfo = true;
 
    static bool hideButton = false;
 
 
    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }
    }
 
public void OnInitialized()
 
    {
 
    }
 
    public void OnTrackablesUpdated()
    {
        if (!m_RegisteredFormat)
        {
            CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
            m_RegisteredFormat = true;
        }
        if (m_LogInfo)
        {
            CameraDevice cam = CameraDevice.Instance;
            Image image = cam.GetCameraImage(m_PixelFormat);
            if (image == null)
            {
                Debug.Log(m_PixelFormat + " image is not available yet");
            }
            else
            {
                string s = m_PixelFormat + " image: \n";
                s += "  size: " + image.Width + "x" + image.Height + "\n";
                s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
                s += "  stride: " + image.Stride;
                Debug.Log(s);
                m_LogInfo = false;
            }
        }
    }
 
    void TakeScreenshot() {
 
        CameraDevice cam = CameraDevice.Instance;
        Image image = cam.GetCameraImage(m_PixelFormat);
        if (image == null)
        {
            Debug.Log(m_PixelFormat + " image is not available yet");
        }
        else
        {
            string s = m_PixelFormat + " image: \n";
            s += "  size: " + image.Width + "x" + image.Height + "\n";
            s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
            s += "  stride: " + image.Stride;
            Debug.Log(s);
 
            Texture2D tex = new Texture2D (image.Width, image.Height, TextureFormat.RGB24, false);
            image.CopyToTexture(tex);
            // tex.ReadPixels (new Rect(0, 0, image.Width, image.Height), 0, 0);
            tex.Apply ();
 
            byte[] bytes = tex.EncodeToPNG();
            Destroy(tex);
 
           // For testing purposes, also write to a file in the project folder
           File.WriteAllBytes(Application.persistentDataPath + "/Screenshot.png", bytes);
 
 
           // now use Unity's built in
           Application.CaptureScreenshot("UnityScreenshot.png");
         }
 
     }
 
     void OnGUI () {
 
         if( !TakePicture.hideButton )
         {
              if(GUI.Button(new Rect(40,80,160,40), "Take Picture"))
              {
                  TakePicture.hideButton = true;
                  TakeScreenshot();
              }
         }
     }
}

Obtain unity camera image (without augmentation)

August 7, 2013 - 4:57am #6

Assets/Qualcomm Augmented Reality/Scripts/TakePicture.cs(5,14): error CS0535: `TakePicture' does not implement interface member `ITrackerEventHandler.OnInitialized()'

To fix this just put in the following empty function:

 

    public void OnInitialized()

    {

    }

 

 

However, the image appears to be kind of flipped or mirrored in some way.

This is exactly what I said in my post:

            Note that you will need to flip this vertically as it is straight from the Camera.

 

... so you will need to scour the Unity forums and various resources to figure out how to do this.

 

HTH

N

Obtain unity camera image (without augmentation)

August 7, 2013 - 4:47am #5

I tried your script in Unity (Version 4.2.0f4) and got the following error:

Assets/Qualcomm Augmented Reality/Scripts/TakePicture.cs(5,14): error CS0535: `TakePicture' does not implement interface member `ITrackerEventHandler.OnInitialized()'

 

I then tried to just remove the offending line from my original script, and I was able to capture the camera image without augmentation on 2 android devices (Galaxy S2,  Xperia A).

However, the image appears to be kind of flipped or mirrored in some way.

(Please refer to the attached picture).

 

Is there a way to fix this?

Since my original script seems to crash on iOS devices, I think it would be better to get the one you send me to compile first.

 

Thank you,

 

Joce

 

 

AttachmentSize
Image icon InvertedImage.png473.02 KB

Obtain unity camera image (without augmentation)

August 7, 2013 - 3:03am #4

Thank you very much for your answer and support.

I will try out the script and provide feedback as soon as possible.

 

Joce

Obtain unity camera image (without augmentation)

August 7, 2013 - 2:21am #3

Hi,

I was able to use the code below to take a screenshot without an augmentation.

It creates two screenshots: 

Screenshot.png - without augmentation.  Note that you will need to flip this vertically as it is straight from the Camera.

UnityScreenshot.png - with augmentation.  Note that this reflects whatever comes out from Unity, which explains the different resolution from the Camera image.

 

When I tried your original code it crashed.  When I tried to integrate your original code with the recommended approach I got a black screen which was due to this offending line:

// tex.ReadPixels (new Rect(0, 0, image.Width, image.Height), 0, 0);

...which actually copies the contents of the currently active framebuffer into the texture - which just happens to be black at the time.

 

N

 

 

using UnityEngine;

using System.Collections;

using System.IO;

 

public class TakePicture : MonoBehaviour, ITrackerEventHandler

{

    private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;

    private bool m_RegisteredFormat = false;

    private bool m_LogInfo = true;

 

    static bool hideButton = false;

 

 

    void Start()

    {

        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));

        if (qcarBehaviour)

        {

            qcarBehaviour.RegisterTrackerEventHandler(this);

        }

    }

    public void OnTrackablesUpdated()

    {

        if (!m_RegisteredFormat)

        {

            CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);

            m_RegisteredFormat = true;

        }

        if (m_LogInfo)

        {

            CameraDevice cam = CameraDevice.Instance;

            Image image = cam.GetCameraImage(m_PixelFormat);

            if (image == null)

            {

                Debug.Log(m_PixelFormat + " image is not available yet");

            }

            else

            {

                string s = m_PixelFormat + " image: \n";

                s += "  size: " + image.Width + "x" + image.Height + "\n";

                s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";

                s += "  stride: " + image.Stride;

                Debug.Log(s);

                m_LogInfo = false;

            }

        }

    }

 

    void TakeScreenshot() {

 

        CameraDevice cam = CameraDevice.Instance;

        Image image = cam.GetCameraImage(m_PixelFormat);

        if (image == null)

        {

            Debug.Log(m_PixelFormat + " image is not available yet");

        }

        else

        {

            string s = m_PixelFormat + " image: \n";

            s += "  size: " + image.Width + "x" + image.Height + "\n";

            s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";

            s += "  stride: " + image.Stride;

            Debug.Log(s);

 

            Texture2D tex = new Texture2D (image.Width, image.Height, TextureFormat.RGB24, false);

            image.CopyToTexture(tex);

            // tex.ReadPixels (new Rect(0, 0, image.Width, image.Height), 0, 0);

            tex.Apply ();

 

            byte[] bytes = tex.EncodeToPNG();

            Destroy(tex);

 

           // For testing purposes, also write to a file in the project folder

           File.WriteAllBytes(Application.persistentDataPath + "/Screenshot.png", bytes);

 

 

           // now use Unity's built in

           Application.CaptureScreenshot("UnityScreenshot.png");

         }

 

     }

 

     void OnGUI () {

 

         if( !TakePicture.hideButton )

         {

              if(GUI.Button(new Rect(40,80,160,40), "Take Picture"))

              {

                  TakePicture.hideButton = true;

                  TakeScreenshot();

              }

         }

     }

}

 

Obtain unity camera image (without augmentation)

August 5, 2013 - 8:03am #2

Hi,

Apologies for the delay in replying.

When I attach your script to the AR Camera in the Image Targets App, and run it on an iPodTouch 5G, it crashes.

Does it work at all for you?  If so, which platforms?

N

 

 

Log in or register to post comments