Log in or register to post comments

Locking rendering and camera FPS

February 4, 2020 - 12:44pm #6

UPDATE: Please note that OpenGLES3 does not suffer from this problem,  issue only happens in Metal.  (which has me thinking this might be a bug and not be working as intended)

When a device (iPad in this case) is rendering too much per frame the performance slows down but the camera feed runs at different frame rate. So things get misaligned.  See this video: https://www.dropbox.com/s/bzui4jh3te0w2sc/KYFX9256.MP4?dl=0  I used the samples and duplicated things, turned on shadows, etc, to simulate a heavy load.  

It seems the camera feed and the rendering exists on different threads.  I understand all the ways to fix this,  but my particular use case it's imperative for things to be in sync even at the cost of frame rate under heavier loads.

Two questions:

1.  Is there some setting I'm missing in Unity that will allow things to stay in sync?  I didn't have any luck changing multithreaded rendering, vsync, etc.

2.  How is the result of GetRecommenedFPS determined?  I assume it's a human being taking into account various factors and inputting in a predetermined database.  Do these values change over time?  Obviously it should to keep up with OS upgrades slowing devices and app experience expectations.



 

Locking rendering and camera FPS

February 5, 2020 - 5:56am #5

Hi,

If this is happening with Metal and not with OpenGLES3 than yes, I would tent to agree that this would be bug but visible only with lower end models or depending on the load.

Coming back to your your questions.

1. I believe you answered it with your Edit

2. As per our API documentation https://library.vuforia.com/content/vuforia-library/en/reference/unity/classVuforia_1_1VuforiaRenderer.html#abe4ffb36f5d6f70ef3612bb39cb41cfe

Query recommended rendering frame rate based on application hints. The target rendering frame rate of an AR or VR application is an important trade-off between optimal experience and device power usage. The choice is influenced by multiple parameters including device type, the active Trackers, the camera and/or sensor frame rates. Furthermore there are application specific trade offs to consider. These hints can be passed to the function as parameters (see FpsHint). For example, an application with animated content may need consistent 60 fps rendering even on a device that can only deliver poses at 30 fps. GetRecommendedFps considers the device parameters as well as the application specific hints and returns a recommended frame rate. The returned value can then be set via setTargetFps. Note that GetRecommendedFps may return different values tuned to the active CameraDevice.CameraDeviceMode and active Trackers. Thus it is recommended to call this API after the application has completed the camera and tracker setup as well as when an application transitions between modes (For example when transitioning between AR to VR modes)

Could you please provide me the unity project to quickly test the behavior?

Also I would be interested to know the device model being used? ( which Ipad version)

Thank you.

Vuforia Engine Support

Locking rendering and camera FPS

February 5, 2020 - 6:57am #4

Thanks for checking this out, won't have time today, will provide a project tomorrow.  It was an iPad Air 2 (latest iOS) (non ARKit).   It also happens on an iPhone 8+ as well (but the load has to be much higher).  I've seen it with Unity 2018.4-2019.3 with Vuforia 8.3.8-8.6.7, but it may be exist outside of that. It happens in all modes: Platform_Fusion, Vuforia_Fusion and Vision_Only.

 

 



 

Locking rendering and camera FPS

February 5, 2020 - 11:33am #3

Ok, found some time today.  Please keep me updated, this issue is important to us.  I didn't mention in the video but this is present in all devices tried and all vision modes.  If you need to replicate on a newer device, you'll have to make the graphics processing workload more intensive by duplicating things, ramping up to 60fps, or some other method.

Video explaining the bug:

https://www.dropbox.com/s/9dvxteow62dc009/Vuforia-Metal-Lag-Bug.mp4?dl=0

Unity project file (2019.3 f6)

https://www.dropbox.com/s/1mxq2wxihq2se6i/Vuforia-Metal-Lag-Bug.zip?dl=0

 

Locking rendering and camera FPS

February 18, 2020 - 7:38am #2

Hi,

Thank you for the video and project.

I tried reproducing it with an iPhone 11 (only iOS available today) and by increasing the count/texture I got the app to crash :). 730 Augmentations with shadows etc.

Tomorrow I will have my hands on an iPad Air, will retest and update the thread.

Thank you.

Vuforia Engine Support

Locking rendering and camera FPS

February 20, 2020 - 10:46am #1

Great, hopefully you can reproduce.  I'm thinking this might be thermal throttling by the OS (device gets warm) and it just handles things differently when it come to Metal.  Another easy way to see this is Application.targetFrameRate = 60; on an older device.

Log in or register to post comments