Log in or register to post comments

ARCamera background texture access on iOS

October 1, 2015 - 9:11pm #1

Hi,

I've got an ARCamera and I need to get access to the background texture. How can I do that? I've checked the background texture access sample, but I was nothing useful in there; just UI handling code, OpenGL error checking and whatever the default ARCamera prefab has. Am I missing something? I've tried this one here but it looks awfully outdated and never gives me any valid image.

I'm working with Vuforia 5.05 and Unity 5.1.3.

Thanks,

Babis

ARCamera background texture access on iOS

October 15, 2015 - 7:05pm #9

So, after plenty of messing around I ended up taking the sample projects (half way down this page: https://developer.vuforia.com/downloads/samples ) and flat out munged things together into something that worked. I couldn't get the scripts in the link to reliably return a constant video stream of what the camera is seeing, and I have no idea why it's working but it seems to be doing so and my aim is to limp along with this solution.

It's a shame we can't set something insainely straightforward and simple like:
mat.mainTexture = CameraDevice.Instance.GetBackgroundTexture();

ARCamera background texture access on iOS

October 15, 2015 - 5:14am #8

This is the link from which you can download the Occlusion Management sample (which contains code to setup and use the camera texture as a Unity Texture2D object and how to handle it in a Shader):

https://developer.vuforia.com/downloads/samples

 

This is the link to the Camera Image Access in Unity (which allows you to access the camera image pixel array):

https://developer.vuforia.com/library/articles/Solution/How-To-Access-the-Camera-Image-in-Unity

 

 

ARCamera background texture access on iOS

October 14, 2015 - 9:26pm #7

I'm having the same issue - using the latest Unity and Vuforia.

I've searched through as many examples as I can find and exhausted hours trying to get something suitable running for iOS devices. The best I can manage is by looking up the CameraDevice Image and filling a texture, but I still need to call Texture2D.Apply to complete the process and get a final image and it's painfully slow.

Is there an actual link to an example scene that I could use to try and find a better solution. The standard link doesn't seem to be up to date, and other pages are down or outdated.

ARCamera background texture access on iOS

October 2, 2015 - 7:47am #6

It's returning null in Play mode - I thought to fix this on the mac before I go on to iOS.. Thanks for the pointer, I'll check that sample out. Does it give info though on how to get the texture from the ARCamera, letting ARCamera operate normally? Note that my intention is to get the video texture and still let the ARCamera output to a rendertexture, so that I can apply later some shader with these two textures as inputs in order to extract the 3D mask, as this is the only (semi-)reasonable way that I've found to extract it.

ARCamera background texture access on iOS

October 2, 2015 - 7:38am #5

Does the camera image returns null in Play Mode (webcam) or on device ?

Note that Play Mode only supports RGBA and GRAYSCALE formats;  on device you would  use RGB888 or RGB565 instead

As far as registering a Texture2D object, this is shown in the Occlusion Management sample app (see the BoxSetupShader.cs script);

the article erroenously refers to the Background Texture Access because in older versions of Vuforia the sample code was also showing how to register a Texture2D;  you can find that code however in the Occlusion Management sample.

 

ARCamera background texture access on iOS

October 2, 2015 - 7:28am #4

Hi Alessandro, 

 

I've done that and image returns null every time, no matter the requested format. Note that I have an ARCamera running, if that makes any difference. Regarding the OpenGL texture way, is that possible using the unity extension? I checked the readme.txt and there was no mention of two ways of doing that. Also, I don't know how to register an OpenGL Texture2D in Unity. I went through the source files and I found no hint of how that's done. Could you give me any pointers?

Thanks,

Babis

ARCamera background texture access on iOS

October 2, 2015 - 1:37am #3

Note about the article:

the QCARBehaviour class refers to Vuforia 4.x; in Vuforia 5.x you'll want to use VuforiaBehaviour.

Also, add this line at the top of the script:

using Vuforia;

 

ARCamera background texture access on iOS

October 2, 2015 - 1:34am #2

The Background Texture Access sample shows how to access the background texture from a shader, and how to manipulate such texture in the shader code.

If you need to access the background camera image information and the pixel buffer (array of pixels), see this;

https://developer.vuforia.com/library/articles/Solution/How-To-Access-the-Camera-Image-in-Unity

 

Log in or register to post comments