Log in or register to post comments

Integrating Vuforia with External Native (C++) Rendering Engine

May 5, 2017 - 12:13pm #1

Hey All,

 

I am attempting to integrate Vuforia with my company's C++ Rendering Engine. Essentially, I want to use Vuforia to encapsulate the camera API and perform the object tracking component of my AR app while I perform all the rendering with our company's engine. Since vuforia supports rendering to an OpenGL texture, I figured I would create an external texture, tell vuforia to use that texture, then after vuforia updated it with the latest camera image, I would pass the texture binding down to my c++ engine to use for rendering the background. This is all done on the rendering thread of my app (via a callback) and looks like this:

        if (!mTextureCreated)

            {

                // create the openGL texture that the camera will render / blit to

                int[] tmp = new int[1];

                GLES20.glGenTextures(1, tmp, 0);

                videoBackgroundTex.videoBackgroundTextureID= tmp[0];

                GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, videoBackgroundTex.videoBackgroundTextureID);

                GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GlTextureWrapS, GLES20.GL_CLAMP_TO_EDGE);

                GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GlTextureWrapT, GLES20.GL_CLAMP_TO_EDGE);

                GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GlTextureMinFilter, GLES20.GL_NEAREST);

                GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GlTextureMagFilter, GLES20.GL_NEAREST);

                mTextureCreated = true;

            }

           

            if (!mTextureLinked)

            {

                // tell vuforia to use the texture we created

                if (!Renderer.getInstance().setVideoBackgroundTexture(videoBackgroundTex))

                {

                    Log.Error("AR-Demo", "Unable to set video background texture");

                    return;

                }

                mTextureLinked = true;

            }

            // update video background tex

            State state = TrackerManager.getInstance().getStateUpdater().UpdateState();

            Renderer.getInstance().begin(state);

            if (!Renderer.getInstance().updateVideoBackgroundTexture())

            {

                Log.Error("AR-Demo", "Unable to update video background texture");

                return;

            }

            Renderer.getInstance().end();

            /* pass texture updated by updateVideoBackgroundTexture to my C++ engine here */

 

This however never seems to actually populate the contents of that texture binding. Every time I bind it to a sampler in our engine the texture looks completely black. So is this method for 'populating' the texture with the camera image invalid? If so, is there a better way to go about doing this?

I also noticed that during a call to Vuforia.OnResume, I recieve an exception with the following error: ""RenderManager: Could not retrieve a valid GLSurfaceView in view hierarchy, therefore cannot set any render mode". In our app, we cannot use a GLSurfaceView because this class not only provides a render target surface but also performs OpenGL context management which interferes with our rendering engine's OpenGL context creation / management. So we instead render to a plain SurfaceView so that we can manage the context ourselves. Does vuforia currently support this kind of workflow or do you have to use a GLSurfaceView?

Any help is appreciated,

 

Alex

Integrating Vuforia with External Native (C++) Rendering Engine

June 2, 2017 - 1:49pm #2

Hi,

It seems the code you are using is from VideoPlayback sample. Have you taken a look into the current samples?

 

Check the SampleAppRenderer.java for android there we use the texture the same way you are trying to use it. Look into renderVideoBackground method inside of the class file.

mRenderer.setTextureUnit and mRenderer.updateVideoBackgroundTexture

 

Those would accomplish what you are looking for. Use GLES20.GL_TEXTURE_2D instead of GLES11Ext.GL_TEXTURE_EXTERNAL_OES

 

 

Log in or register to post comments