Log in or register to post comments

Getting the bytes from Vuforia...

June 13, 2012 - 2:40am #1
	public IEnumerator Snapshot () {
               print ("Vuforia: snapshot");
		yield return new WaitForEndOfFrame();
		
		Image.PIXEL_FORMAT mPixelFormat = Image.PIXEL_FORMAT.RGB565;
		Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);
		
		print("Vuforia: reading bytes");

		
		byte[] bytes = cameraImage.Pixels;

		
		print ("Vuforia: bytes read");

}

... so, oddly enough, everything shows up in debug log, except the "bytes read" print never occurs... how do you get the bytes?

Getting the bytes from Vuforia...

September 19, 2012 - 2:17pm #19

@ina - which image format are you seeing that result with?

Getting the bytes from Vuforia...

September 17, 2012 - 1:43am #18

Sorry, I don't need to ask this anymore. Please delete my post. Smile

Getting the bytes from Vuforia...

July 7, 2012 - 6:42pm #17

Hi DavidBeard, thanks for the overview but I am not sure how bytes[] returns a byte array that is 768000 in length ... for 960x480 resolution screen (960*480 = 460800)

What resolution difference is Vuforia taking from the screen resolution? 

Getting the bytes from Vuforia...

July 5, 2012 - 8:39am #16

OnTrackablesUpdated is a callback registered w/ QCARBehaviour as a TrackerEventHandler.
This is called within QCARBehaviour's Update method as such..

// Updates the scene with new tracking data. Calls registered
// ITrackerEventHandlers
void Update()
{
if (!mIsInitialized)
return;

// Get the current orientation of the surface:
ScreenOrientation surfaceOrientation = (ScreenOrientation)getSurfaceOrientation();

// Check if we need to update the video background configuration and projection matrix:
if (QCAR.IsRendererDirty() || mProjectionOrientation != surfaceOrientation)
{
ConfigureVideoBackground();
UpdateProjection(surfaceOrientation);
}

// Bind a simple material to clear the OpenGL state
mClearMaterial.SetPass(0);

// QCARManager renders the camera image and updates the trackables
QCARManager.Instance.Update(mProjectionOrientation);

// Tell Unity that we may have changed the OpenGL state behind the scenes
GL.InvalidateState();

// Update the camera clear flags
UpdateCameraClearFlags();

// Let the trackable event handlers know that all trackables have been updated
foreach (ITrackerEventHandler handler in mTrackerEventHandlers)
{
handler.OnTrackablesUpdated();
}
}

You'll notice that the OnTrackablesUpdated call occurs after the camera image has been updated. That's the critical aspect, it ensures that there is a complete Image frame available. So no, it's not mandatory to obtain the bytes in OnTrackablesUpdated, but it's probably the best approach.

If you'd like to trigger the Image capture interactively, you could set a flag on the event handler for this purpose - i.e. the Image is only obtained for when the flag is set.

Getting the bytes from Vuforia...

July 5, 2012 - 2:36am #15

DavidBeard wrote:

Please try this script, on the ARCamera, and if that doesn't work I'll send you a scene to test. public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler { private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888; private bool m_RegisteredFormat = false; private bool m_LogInfo = true; void Start () { QCARBehaviour qcarBehaviour = (QCARBehaviour)FindObjectOfType (typeof(QCARBehaviour)); if (qcarBehaviour) { qcarBehaviour.RegisterTrackerEventHandler (this); } } public void OnTrackablesUpdated () { if (!m_RegisteredFormat) { CameraDevice.Instance.SetFrameFormat (m_PixelFormat, true); m_RegisteredFormat = true; } if (m_LogInfo) { CameraDevice cam = CameraDevice.Instance; Image image = cam.GetCameraImage (m_PixelFormat); if (image == null) { Debug.Log (m_PixelFormat + " image is not available yet"); } else { string s = m_PixelFormat + " image: \n"; s += " size: " + image.Width + "x" + image.Height + "\n"; s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n"; s += " stride: " + image.Stride + " Pixel Count = " + image.Pixels.Length; Debug.Log (s); m_LogInfo = false; } } } }

 

Must the call be put in OnTrackablesUpdated ? 

 

What triggers OnTrackablesUpdated? 

Getting the bytes from Vuforia...

July 3, 2012 - 1:30am #14

@ina Sign up, when you did to work it. I would be very happy about it :)

Getting the bytes from Vuforia...

July 2, 2012 - 6:29pm #13

Please try this script, on the ARCamera, and if that doesn't work I'll send you a scene to test.

public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
{
private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
private bool m_RegisteredFormat = false;
private bool m_LogInfo = true;

void Start ()
{
QCARBehaviour qcarBehaviour = (QCARBehaviour)FindObjectOfType (typeof(QCARBehaviour));
if (qcarBehaviour) {
qcarBehaviour.RegisterTrackerEventHandler (this);
}
}

public void OnTrackablesUpdated ()
{
if (!m_RegisteredFormat) {
CameraDevice.Instance.SetFrameFormat (m_PixelFormat, true);
m_RegisteredFormat = true;
}

if (m_LogInfo) {
CameraDevice cam = CameraDevice.Instance;
Image image = cam.GetCameraImage (m_PixelFormat);

if (image == null) {
Debug.Log (m_PixelFormat + " image is not available yet");
} else {
string s = m_PixelFormat + " image: \n";
s += " size: " + image.Width + "x" + image.Height + "\n";
s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
s += " stride: " + image.Stride + " Pixel Count = " + image.Pixels.Length;
Debug.Log (s);

m_LogInfo = false;
}
}
}
}

Getting the bytes from Vuforia...

July 2, 2012 - 5:57pm #12

@DavidBeard: Is it possible that you provide a .cs or scene file for how you managed to get a non-null byte stream? 

 

DavidBeard wrote:

Have you confirmed that you're getting a null? When are you calling Snapshot? Try calling this from LateUpdate(). http://unity3d.com/support/documentation/ScriptReference/MonoBehaviour.LateUpdate.html

Getting the bytes from Vuforia...

July 2, 2012 - 2:15am #11

It is unfortunately not so easy for Android and iOS. First, i must flip the picture horizontally. But for me it works not yet anyway and it is very slow. :( It works with the laptop webcam, but not with Android or iOS.

Getting the bytes from Vuforia...

July 1, 2012 - 2:24pm #10

Russel wrote:

Works ZXing in Unity for Android without Vuforia? How did you do that? Have you take this solution? http://ydaira.blogspot.de/2012/05/how-to-decode-qr-code-using-unity3d.html

I did not get it to work. :(

@Russel It is very simple to get zxing to work by itself. Download the .cs file included in link above. Download the .dll file and put it in Plugins folder. Put the .cs file inside a game object. Then press play. Point your webcam at a QR Code. In Debug Console you will see the QR code decoded into a string. 

For Android device deployment, you can just assign the print string to a guiText, or check logcat. 

 

@DavidBeard However, I am apparently not even able to get the byte stream from Vuforia video to make zxing read from Vuforia to integrate the two! (Always null) I don't think it's a device issue... are you actually able to read the video stream from Vuforia, or just providing snippets in theory?

Getting the bytes from Vuforia...

July 1, 2012 - 2:17pm #9

To be clear, I cannot get zxing AND Vuforia to work together, ie, for zxing to read from Vuforia's video stream. 

 

I can get each individual library to work though separately 

Getting the bytes from Vuforia...

June 29, 2012 - 4:56am #8

Works ZXing in Unity for Android without Vuforia? How did you do that? Have you take this solution? http://ydaira.blogspot.de/2012/05/how-to-decode-qr-code-using-unity3d.html

I did not get it to work. :(

Getting the bytes from Vuforia...

June 25, 2012 - 12:47am #7

Evo 3D ... The zxing works by itself and Vuforia works by itself of course. 

 

I am wondering if I am just completely calling the bytes incorrectly? Can you provide a snippet for other lines that should be in the code and placement?, ie WaitForEndOfFrame (etc)? 

Getting the bytes from Vuforia...

June 21, 2012 - 8:18pm #6

Which devices are you testing this on?

Getting the bytes from Vuforia...

June 21, 2012 - 5:48pm #5

Even calling the function in Late Update, I always get a null... 

Re: Getting the bytes from Vuforia...

June 13, 2012 - 3:40pm #4

Have you confirmed that you're getting a null? When are you calling Snapshot?

Try calling this from LateUpdate().

http://unity3d.com/support/documentation/ScriptReference/MonoBehaviour.LateUpdate.html

Re: Getting the bytes from Vuforia...

June 13, 2012 - 12:26pm #3

Is there a way to wait for a non-null image or to force fetch the last non-null image? I had assumed seeing the vid camera stream means there would be an image for sure

Re: Getting the bytes from Vuforia...

June 13, 2012 - 9:36am #2

That's the correct method, but you should check for a null value.\

Image cameraImage = CameraDevice.Instance.GetCameraImage(mPixelFormat);

        if (cameraImage != null)
        {
            bytes = cameraImage.Pixels;

        }

GetCameraImage will return null for an invalid image..

    public Image GetCameraImage(Image.PIXEL_FORMAT format)
    {
        // Has the format been requested:
        if (mCameraImages.ContainsKey(format))
        {
            // Check the image is valid:
            Image image = mCameraImages[format];
            if (image.IsValid())
            {
                return image;
            }
        }

        // No valid image of this format:
        return null;
    }
Log in or register to post comments