UPDATE: Please note that OpenGLES3 does not suffer from this problem, issue only happens in Metal. (which has me thinking this might be a bug and not be working as intended)
When a device (iPad in this case) is rendering too much per frame the performance slows down but the camera feed runs at different frame rate. So things get misaligned. See this video: https://www.dropbox.com/s/bzui4jh3te0w2sc/KYFX9256.MP4?dl=0 I used the samples and duplicated things, turned on shadows, etc, to simulate a heavy load.
It seems the camera feed and the rendering exists on different threads. I understand all the ways to fix this, but my particular use case it's imperative for things to be in sync even at the cost of frame rate under heavier loads.
Two questions:
1. Is there some setting I'm missing in Unity that will allow things to stay in sync? I didn't have any luck changing multithreaded rendering, vsync, etc.
2. How is the result of GetRecommenedFPS determined? I assume it's a human being taking into account various factors and inputting in a predetermined database. Do these values change over time? Obviously it should to keep up with OS upgrades slowing devices and app experience expectations.
Great, hopefully you can reproduce. I'm thinking this might be thermal throttling by the OS (device gets warm) and it just handles things differently when it come to Metal. Another easy way to see this is Application.targetFrameRate = 60; on an older device.