"We offer new support options and therefor the forums are now in read-only mode! Please check out our Support Center for more information." - Vuforia Engine Team

getcameraimage and more

Hi! I notice in the CameraDeviceBehaviour there seems to be a method to GetCameraImage. Do you have any specific info regarding how to call this method, and what you would expect to get back. More specifically would the image include any Unity-rendered stuff or is it just the video signal from the camera? On a completely different note, I have been browsing around the code and I have noticed a couple of things. You seem to rely on the class constructor to initialize stuff at times. I think the Unity docs discourage this usage and the best practice is to use Start or Awake to initialize things. The other thing that could be improved is to avoid making repeated calls like FindObjectOfType, but instead cache the return values for speed. This is probably not a huge factor for performance, but it would certainly influence it. Since the poor old device is fighting for its life, performancewise, we might want to give it as much room as we can. One thing that I noticed is that in adb logcat the frametimes seem to vary in an unusual way. I guess this has to do with the AR plugin workload varying over time. What can be done to lessen the load on the plugin? My guess is to have as few concurrent fiducials as possible, make sure the lighting conditions are very good, and the fiducial quality as well, and to use autofocus. TIA Niklas Wörmann

The camera image returned by that method is the raw pixel buffer from the camera (does not include any augmentation). Here is a bit of sample code for using it. Note that this is an implementation of the ITrackerEventHandler.

fjhamming

Mon, 06/27/2011 - 10:31

I tried to implement your code suggestion to apply an environmental relection on a model with a reflection shader. But converting the pixels to a texture2d gives me some trouble.

pixelplacement

Sat, 07/30/2011 - 04:57

I'm running into the same image with being unable to render the texture. IsValid() even returns true so maybe it has something to do with Unity's LoadImage()?

fjhamming

Sat, 07/30/2011 - 08:11

After some experiments i managed to create a new texture2d, and use the setPixels function on the that texture.

Unfortunately the Texture2D api doesn't provide a convenient way to create a texture given a raw pixel buffer (as far as I know). The LoadImage method expects a jpg or png formatted byte array, which is different than a simple pixel buffer.

pixelplacement

Mon, 08/01/2011 - 17:11

Hmmm... we REALLY need a fast technique to grab the video stream for effects and sampling. Anyone up for tackling an approach?

adrianrdzv

Tue, 08/23/2011 - 08:36

Any update on this topic? It would be very useful to use the camera image in textures for improving augmentation.

WreckCenter

Mon, 10/10/2011 - 23:25

i'm moving a non-unity qcar project to unity that involves using the camera image and rendering to an off screen buffer with a pixel shader...that should be about the fastest way to do this no? I've got the whole thing working outside of unity, just need to hook it up to qcar/unity

I would like to add a vote for QCAR adding a fast build in solution to grab the video texture from the camera in Unity3d.

WreckCenter

Wed, 11/09/2011 - 16:52

is there a way to get at the cameraimage in the native code? my knowledge of unity cameras and rendertextures isnt that great. i had some color tracking code working in an ios project using QCAR, and now our project is in Unity.

[QUOTE] I figured i can send the image info via a native dll call, and use my old code [/QUOTE] This is the officially supported method. There's something you can try, though, that I've just thought of.

WreckCenter

Thu, 11/10/2011 - 15:26

Hi Kim, I actually had all this working outside of Unity, so know about the callback, just didn't think to consider still using it while also using the Unity plugin.

[QUOTE] in the unity script is where i registered for the image format i want, would that carry over to the update callback?