Hi Indigo,

I got the same problem as you, did you manage to fix it?

Thanks

Hi everyone,

This is very simliar to an issue I'm currently facing (integrating Vuforia into my own engine's render cycle). I think I've got everything setup, except I'm running into one odd issue.

To hook the camera feed in I am attempting to render the camera feed into an OpenGL texture that I then apply to an object within my engine. To do this I create a new OpenGL texture and set it to be rendered to using `QCAR::Renderer::getInstance().setVideoBackgroundTextureId(textureId);`. Each frame the texture becomes progressively brighter (extremely fast, if I can confirm this by laying the camera onto a flat surface so it receives little light each frame), until it is completely white. Currently this occurs with the following render cycle (for my Vuforia section):

renderer.begin();

drawElements using the textureId from the setVideoBackgroundTextureId call.

renderer.end();

If I remove renderFrameQCAR from my UIView (testing on iOS, writing the handling code in C++), which is a `No op`,  nothing renders to the texture. If I call `drawVideoBackground()` nothing renders to the texture. If I call `bindVideoBackground()` and then use that texture slot I get the same results as using my own texture. If I use `drawVideoBackground()` and `bindVideoBackground()` together I get the same result, except the camera feed only fills the bottom left quarter of the texture.

Any help would be greatly appreciated.

Thanks in advance

Indigo

This is a high-level view of the OpenGL commands involved in the videobackground rendering (as per "drawVideoBackground"):

• glActiveTexture( GL_TEXTURE0 )
• glBindTexture ( some_texture_id )
• glDisable ( GL_DEPTH_TEST );
• glDisable (GL_CULL_FACE );
• glEnableVertexAttribArray ( some_array ) / glDisableVertexAttribArray ( )
• glViewport ( some_viewport_coords )
• glUseProgram ( some_shader_id )

If you counter-act each of those by setting your own texture unit, texture, shader (program) and vertex arrays, you should be fine.

Note that some 3rd party engines might internally do things like "clearing the color buffer" ( glClear) and/or the depth buffer, which would result in clearing the video background away and replacing it with the defined clear color (in this case, you will want to prevent your 3d engine from doing that).  

Thank you Alessandro. The thing is that I managed for some time to solve the integration of Vuforia with the 3rd party 3D engine. Of course, I dealt with shader program binding, and correct uniform setup, and the only change I had to do was to re-enable the vertex attrib pointers in the 3D engine: enableVertexAttribArray. But now, I am once again having issues. Wouldn't it be possible to have a glimpse to everything that Vuforia's drawVideo call is doing render-wise? This is like looking for a needle in a haystack blindfolded.

...The states listed in the https://developer.vuforia.com/resources/dev-guide/opengl-state-changes-video-background-renderer post, are definitively not enough as modifications to vertex buffers and attributes are also made by the video drawing routine from Vuforia....

The vertex buffers and attributes must be passed to the graphics pipeline using glEnableVertexArray and glVertexAttribPointer and via the glUniform commands; this is however a general must-do requirement for any piece of code using OpenGL ES 2.0 (which is pretty much shader-based and has very few "states" in the traditional sense);

similarly, you also need to make sure your own Shader is binded ( via  glUseProgram( ... )  ) after drawing the videobackground and anyway before drawing your vertex arrays;   this is regardless of the OpenGL state changes.....; as you know, in OpenGL ES 2.0 many of the OpenGL states that you would find in OpenGL ES 1.1 are simply handled via GLSL shader logic, and you typically control them via Uniforms and vertex attributes that you pass to the shader from the application code.

Given that the drawVideoBackground uses its own shader, while you obviously need to use your own shader for your geometry, there is no possible conflict between the attributes and uniforms used by Vuforia with the ones used with your own shader, as long as you remember to:

- bind your shader

- set your uniforms

- call glVertexAttribPointer and pass your own vertex arrays and attributes with it

Hello,

All this information is super useful. I also managed to control the rendering of the video on my own and integrate my own engine. But I am having some problems reseting the states both for my engine rendering after the video draw and back to vuforia so it can render the video correctly.

Would anyone mind to post some code snippet on how to do this correctly? What kind of states should be reset and how in both scenarios?

For me, the perfect scenario would be:

1.- Store all the states that will change for the video in the background drawing.

2.- Draw the video in the background.

3.- Restore all the states stored in point 1.

Is there any code to do something like this? The states listed in the https://developer.vuforia.com/resources/dev-guide/opengl-state-changes-video-background-renderer post, are definitively not enough as modifications to vertex buffers and attributes are also made by the video drawing routine from Vuforia.

Thank you very much in advance!

Hi stageev:

Is it possible to turn documentation into a wiki?

thanks for the suggestion; we don't have specific plans for that at the moment, but we could take it into consideration.

Yes, i was wondering. Is it possible to turn documentation into a wiki? That will be so much better.

Thanks for the link, very usefull. I must have missed that.  I wished someone had given me the link when I first started this thread.

@fencingfleas and stageev: Just a little tip on the OpenGL state: the state changes made by Vuforia are described in the Dev Guide:

https://developer.vuforia.com/resources/dev-guide/opengl-state-changes-video-background-renderer

Oh yes, forgot to mention the state problem. Thanks.

So I managed to make it work, like Stageev mentioned just run your own loop and call begin(), drawVideoBackground() and end() where appropriate. I did exactly the same on both iOS and Android platforms. There is one thing that you should be carefull with though. If you have optimized your graphics engine and have OpenGL state management system, make sure that you reset the opengl states back after vuforia drawVideoBackground(). Also I noticed vuforia doesn't set it's opengl states properly before initiating render calls. (e.g. Vuforia rendering would be messed up if your renderer turns on second texture stage and does multi-texture blending. Vuforia doesnt turn off second texture index before initiating it's render calls). So if you have problem rendering camera background, make sure you reset the opengl states back to what vuforia is expecting them to be. This one took me a while to figure out.

The same how you would do it in renderFrameQCAR, you use QCAR::Renderer::getInstance().drawVideoBackground(); Nothing really changes except the fact that you use your own render loop.

Yeah i have an application which runs it's own render loop. You don't really have to use the one provided by QCAR and in complex cases it's even harmful to use it. In cases where you integrate vuforia into an existing engines it's the way to go.

And specifically on iOS it's probably always better to use your own render loop because creating it with CADisplayLink is extremely easy and it gives more flexibility.

Hi stageev,

I am curious to know that if you are avoiding renderFrameQCAR completely oniOS, how are you displaying the camera feed?

Have you managed to integrate it successfully?

cheers

N

Can't say anything about android version, but in iOS you can safely ignore renderFrameQCAR method and as ignore i mean remove it. Vuforia will usually go through your uiviews searching for one implementing UIGLViewProtocol which defines this method and if it can't find it it will not attempt to call it, so you can safely use your own rendering loop in the same manner like you use renderFrameQCAR.

Hi, we don't have a tutorial on how to integrate Vuforia with 3rd party engines (as in general it depends very much on the specificities of each 3D engine);

here are a couple of hints:

- the OpenGL rendering in the samples (e.g. ImageTargets) is done in C++ and is called from the GL renderer specified in Java code (see onDrawFrame() method in ImageTargetsRenderer.java for instance);  so, if you can hook your engine into the Java renderer, then you might be able to perform your own rendering 

- there are some threads in various Android-related forums/blogs that talk about how to control OpenGL rendering; this is one example:

http://stackoverflow.com/questions/7421976/opengl-es-calling-ondrawframe-manually

But there are many others.

Also, if you search the Forum, other developers have successfully made integration of third party engines (e.g. JPCT, Adobe AIR,...); you could find some useful information there as well.