Switch between Image Target and Ground Plane

I have a 3D model and I'd like to have to option to attach it to an Image Target OR a Ground Plane. The way I implemented this is by keeping the model as a child of the Image Target, and pressing a button it's parented to the Ground Plane Stage. Pressing the button again brings the model back into the image Target.

It works, but the model is VERY unstable and jittery, sometimes while being parented to the Image Target it behaves like it's in a mid air stage (stays in the same real-world point). I have tried different approaches, even having a copy of the 3D model in both trackables, unregistering targets, nothing seems to work. Am I missing something?

I saw there is a lengthy discussion about a similar problem and it looks like Image Target and Ground Plane don't work well together out-of-the-box, while I had no problems making them work independently (i.e. a scene with only Image Targets or only one Ground Plane)

I'm using Unity 2019.2.5f1 and 2019.2.9f1, this happens in-editor on Windows 7 and the last Mac OS using the webcam and on an iPad Air 2 (no ARKit), haven't tested on other devices.