Log in or register to post comments

Optimisation questions

May 2, 2012 - 10:49am #1

Good day,

We are currently using another AR framework on our product, however we are currently working on porting our product to Vuforia.

The general feeling with Vuforia seems to be better: more stable, tracking seems to be more accurate and more fluid. However, regarding the performances we are suffering from an additional 15 to 30 ms more.

So, since we don't have acces to source code of the binaries I started to look into the code wrapping them. Doing that, the following in QCARManager::UpdateCameraFrame intrigued me:

       ...

        foreach (Image image in CameraDevice.Instance.GetAllImages().Values)
        {
            IntPtr imagePtr = new IntPtr(mImageHeaderData.ToInt32() + i *
                   Marshal.SizeOf(typeof(ImageHeaderData)));
            ImageHeaderData imageHeader = (ImageHeaderData)
                Marshal.PtrToStructure(imagePtr, typeof(ImageHeaderData));

            ...

            // Reallocate if required:
            if (imageHeader.reallocate == 1)
            {
                ...
            }
            else if (imageHeader.updated == 1)
            {
                // Copy data:
                image.CopyPixelsFromUnmanagedBuffer();
            }

            ...
        }

Image::CopyPixelsFromUnmanagedBuffer seems to copy the unmanaged image buffer retrieved by the unmanaged code and make a local copy. This operations sounds to be costly to me. Moreover I don't really see the point of copying this buffer since we can do all the work using OpenGL in the unmanaged code. I've been doing it for my own AR framework, by passing the texture id from unity to unmanaged code and sending directly the texture to the display card.

Curious, I tried to comment this line and surprisingly everything was still working properly. Looking a bit deeper in the code I didn't saw anything using that managed buffer so what's the point of this operation?

(iOS)
Moving on, past week I've been trying to find more general optimisations when I felt on this article that was saying by disabling USE_OPENGLES20_IF_AVAILABLE in the xCode project could increase considerably the performances. So I tried it, when Vuforia disabled it increased the performances by 2 fps however when Vuforia is enabled I've observed a 10 fps gain. However this is highly mitigated since I had no video feedback from the camera in the game. This seems to be caused by the following error:
OpenGLES error 0x0502 in .../builds/iOS/Classes/iPhone_GlesSupport.cpp:204

Which is contained in ::PreparePresentSurfaceGLES generated function:

void PreparePresentSurfaceGLES(EAGLSurfaceDesc* surface)
{
#if GL_APPLE_framebuffer_multisample
	if( surface->msaaSamples > 0 && _supportsMSAA )
	{
		Profiler_StartMSAAResolve();

		UNITY_DBG_LOG ("  ResolveMSAA: samples=%i msaaFBO=%i destFBO=%i\n", surface->msaaSamples, surface->msaaFramebuffer, surface->framebuffer);
		GLES_CHK( glBindFramebufferOES(GL_READ_FRAMEBUFFER_APPLE, surface->msaaFramebuffer) );
		GLES_CHK( glBindFramebufferOES(GL_DRAW_FRAMEBUFFER_APPLE, surface->framebuffer) );

		GLES_CHK( glResolveMultisampleFramebufferAPPLE() );

		Profiler_EndMSAAResolve();
	}
#endif

	// update screenshot here
	if( UnityIsCaptureScreenshotRequested() )
	{
		GLint curfb = 0;
		GLES_CHK( glGetIntegerv(GL_FRAMEBUFFER_BINDING, &curfb) );
		GLES_CHK( glBindFramebufferOES(GL_FRAMEBUFFER_OES, surface->framebuffer) );
		UnityCaptureScreenshot();
		GLES_CHK( glBindFramebufferOES(GL_FRAMEBUFFER_OES, curfb) );
	}

#if GL_EXT_discard_framebuffer
	if( _supportsDiscard )
	{
		GLenum attachments[3];
		int discardCount = 0;
		if (surface->msaaSamples > 1 && _supportsMSAA)
			attachments[discardCount++] = GL_COLOR_ATTACHMENT0_OES;

		if (surface->depthFormat)
			attachments[discardCount++] = GL_DEPTH_ATTACHMENT_OES;

		attachments[discardCount++] = GL_STENCIL_ATTACHMENT_OES;

		GLenum target = (surface->msaaSamples > 1 && _supportsMSAA) ? GL_READ_FRAMEBUFFER_APPLE: GL_FRAMEBUFFER_OES;
		if (discardCount > 0)
		{
			GLES_CHK( glDiscardFramebufferEXT(target, discardCount, attachments) );
		}
	}
#endif
}

I've been trying to debug this line but since I don't have a good OpenGL background I didn't managed to make it work. So I was hoping that you guys might have a better experience of this. From what I found on internet about this error, the only thing I can conclude is that one of the parameters is invalid.

Thanks for your enlightenment and your help in advance!

Re: Optimisation questions

May 15, 2012 - 6:58am #2

Hi scadieux,
I'm running AR Qualcomm with Unity and an iPad3 (performances on iPad3 are so great).
I've in my project two "4stars" trackables, and with the profiler the AR thread takes 30ms constant.
If I want at least 25 fps it left only 10ms for 3d rendering and scripting...which is very short. And this is on iPad 3 ! I can't imagine running my app on iPhone 3GS or an old android.
I'll try to comment the same line in the function you're talking about and change the camera to OPTIMIZE_SPEED, but I'm afraid about AR performances...

Log in or register to post comments