Model tracking on various devices

I'm posting this here in the hope that my frustration will be of use to others.

Some background.  I have a set of dioramas. I have 6 total but only used one for this test.

I am trying to place augmented content on them.  Model tracking seemed like a good fit. 

Here is the result of me trying many different tablets(and one phone) to suss out the best way to use the model tracking.

The model I was trying to track can be seen in the link above, it is a mountainscape setting made with foam and foil and resin for the water.  It is about 3 feet across and 10 inches at the tallest.  I used a Structure Sensor 3D scanner to get the models into the Model Target Generator.

I set it up as a Model Target in Unity with a single 2D guide view.  I tried model recognition, but had some issues so I left it out for the purposes of this test.

My torture test for tracking was to flap the device like a fan until it either couldn't recover tracking or went back to guide view mode.

Galaxy Tab S2:  Surprisingly, it works.  If you move slowly enough the tracking seems to hold its own and when it loses it in fast motion, it comes back easily enough.  The framerate is terrible though, much worse than a multi-target. Lost tracking after just one flap but sometimes it realigns even after showing the guide view and often would realign even if not in the guide view position, i.e. on one side or the other.  Both of these traits were consistent among the Samsung tablets.

Galaxy Tab S3:  Getting tracking to be accurate can be difficult and may require resetting and realigning with the guide view multiple times.  Recognizes relatively easily(aside from having to reset it) and tracks accurately.  Causes large drop in background video FPS but not as bad as the S2.  Similar results in the torture test as the S2.

Galaxy Tab S5e:  Guide view calibration takes many attempts to get to a decent state.  Seems more resistant to motion than the S8 phone.  Background video stuttered somewhat but the content seemed fine.  Similar results in the torture test as the S2 but maybe better realignment.  They do funny things with model names these days.  The S4 is actually more powerful but I don't have one to test.  I'd guess it would be better, but not sure how much.

Galaxy S8 phone:  Detection is similar to the tablets: easy to recognize but with lots of resetting the guide view.  Tracking is decent when it does get set up right.  Framerate remains acceptable in both background video and content.

iPad Air 2:  Hardest to get it to show something.  Worked once or twice but not after that.  When the tracking did work it seemed similar to the 6th gen.  Doesn't have or doesn't support a 60fps camera so video quality looked very similar to the Galaxy tablets.

iPad 6th Generation(2018):  Very difficult to get it to recognize the model.  Took a long time and much fussing.  Once tracking though, it was stable.  Lost tracking during the torture test but re-acquired without having to go back to guide view.  4-5 flaps were enough to break it and force a guide view reset.  Has a 60fps camera which makes the experience much more fluid.

iPad Pro 9.7 inch  Recognizes and tracks well.  Much easier recognition than the 6th gen and maybe slightly better stability and re-acquisition after torture test.  5 flaps were enough to break the alignment enough that I had to reset to the guide view.  When tracking model targets is starts smooth but after about 10ish seconds there are high amounts of camera lag.  Doesn't seem to affect the augmented content but makes moving the tablet unpleasant. Has a 60fps camera and strangely the framerate of the camera doesn't seem to be affected either when it starts lagging.  It just feels like your tablet is on a big spring.

iPad Pro 12.9 inch(2nd Generation)  Rock solid.  The easiest guide view alignment by far and most stable tracking.  Torture test resulted in misalignment that fixed itself within a less than a second.  After 15 or so flaps the content became so misaligned that it didn't recover until I reset the guide view.  Famerate on both the background video and the augmented content seemed entirely smooth throughout.  60fps camera supported.

Conclusions:

Overall the Samsung tablets are much more likely to do something but that something is more likely to be inaccurate.  They often went back to the guide view when things got crazy whereas the iPad's mostly did not bring the guide view back and still kept trying(and failing) to track the model.

All of the Android devices have the benefit of not requiring a Mac.  It was a long feedback loop since the only way to make trackable models is through the Model Target Generator on a Windows machine and the only way to build to an iPad was with the Mac.  I would generate the models in Windows, bring them into Unity, push my project to the Unity collaborate server, download on the Mac and build to the iPad.

Overall the iPad's seem like they have better tracking, maybe due to ARkit being easier to implement on a smaller range of devices.  The iPad Pro 9.7 would be about perfect if it weren't for the background video lag.  I imagine the iPad Pro 10.5 would do as well as the 12.9 since it has the same internals but I didn't have one to test.

The workflow was painful when I was trying to do multiple diroramas with multiple guide views.  I spent hours and hours running various combinations through the Model Target Generator and in the end it wasn't really workable.  I think the model recognition has a hard time with the dioramas because they all look kinda similar, especially from the edge on.  It would often give me the totally wrong model as a guide view, even when I cut it down to 3 models with one guide view each.

I also noted in my multiple target tests that the best guide view for detection is a side view, edge on to the model.  This was easy for it to detect, but the tracking was often all over the place.  I ended up going to a slightly higher view but noted that even then detection would often only happen at a super low angle.

I am probably not going to use model tracking in the end simply due to the fact that the guide view is too difficult for end users to understand.  I will likely end up going for some sort of multi-target/cylinder target that I will fit into the theme of the project.  They don't offer as good of tracking, but the recognition is so much simpler.