"We offer new support options and therefor the forums are now in read-only mode! Please check out our Support Center for more information." - Vuforia Engine Team

get camera feed performance issue

By TStb on Thu, 01/10/2019 - 14:59

Hi,

I am trying to get the camera feed while using Vuforia. Is there a better, especially less CPU consuming, way to get this than CameraDevice.Instance.GetCameraImage(pixelFormat).CopyToTexture(texture2D)?

It impacts the frame rate of my application quite significant (about 45fps drop), since I cannot run it on another thread and it's resolution is 856px x 504px, which is to be expected, but can not be lowered.

Have you given AsyncGPUReadback a shot yet?  https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.Request.html
This example on github is a good place to start: https://github.com/keijiro/AsyncCaptureTest

I did not know about this function, but it appears to need a Texture or a ComputeBuffer as source, which would be great, if I had either. 

Is there a reason you can't use the video background texture?  Ah, duh, you're on Hololens, I see why now.
Maybe Vuforia still retains the video background texture in memory, maybe make sure of that before totally discounting this.  
 

I figured that it is less about getting the data but actually showing it. I can get the raw data with CameraDevice.Instance.GetCameraImage(pixelFormat).pixels without much delay since nothing gets copied.

And you can't use the built in live streaming hololens features cause Vuforia needs access to the camera right?  
You might be hitting the technical limitations of things at this point.

Hello,

Based upon experiments with HoloLens while running Vuforia, here is what we know about the Mixed Reality Capture (MRC) feature:

So I implemented a reasonable running live preview, but I have a problem regarding the alignment between the virtual elements and the video feed. Running Vuforia on my laptop I noticed that the VuforiaBackground is using some scaling and translation for proper alignment.