Log in or register to post comments

MultiTarget trackable flipping orientation for a specific image target

February 14, 2017 - 10:13am #1

Hello fellow developers!

I'm currently struggling with using MultiTarget trackables. For an unknown reason it seems like one of the parts in my MultiTarget is causing the whole trackable to flip constantly in a unexpected direction when it's the main part visible by the camera.

And I know there is a lot of text below, but I'm trying to make it super clear so you are able to understand what's going on.

Context

First a bit of context will help to understand what I'm trying to achieve. Feel free to skip to the next section if you don't care :)

I'm working on the prototype of a commercial app in which power user will be able to re-arranges a predefined set of markers in a given location on a daily basis. Everyday the positioning will be slightly different and thus will need the MultiTarget to be regenerated.

I've wrote a calibration tool that takes a dataset with all the image targets that'll be used for a given scene and then generate a multitarget based on the location of those image targets in the environment. The calibration is saved in form of a dataset that uses the same binary (.dat) but with an enhanced definition (.xml) that now includes the generated multi target.

The calibration is then uploaded to a server which the users will be downloading it from during gameplay.

I've experience this issue both with Vuforia v6.0.117 and v6.2.6, but I've been doing most of my debugging with v6.2.6.

I'm using Unity v5.4.3p1.

MultiTarget Setup

To give you an idea of what my development setup look like, I took a cardboard box and I taped it so it stays in a L shape. Then I putted 2 marker at the bottom and 2 markers at the top.

Have a look to [setup.png], you can see the 4 image targets the multi target is composed of.

[setup.png]

Notes:

  • Those images were layed out by the MultiTargetBehaviour of a MultiTarget I generated with my calibration. And yes their layout correspond to the physical layout.
  • The lines point to the bottom left corner of each image target.
  • The semi-transparent mesh around the image targets represent the cardboard box.

Flipping

To demonstrate the flipping, what I did is a close up of each part of the multi target using the camera. And then slowly move from one part to the next part.

The following parts worked as expected:

  • carpet [carpet-ok.png]
  • poster [poster-ok.png]
  • portrait [portrait-ok.png]

[carpet.png][poster-ok.png][portrait-ok.png]

 

However the tarmac part caused the trackable to flip: [tarmac-flip.png].

[tarmac-flip.png]

Troubleshooting

I've spent a bit more than a day on this issue, and during this time I've looked closely to the differents parts that would explain this behaviour, I've tried:

  • Changing the image used of the image target that the bogus part is using.
    • Initially thought that might be because the marker I was using (tarmac provided by vuforia) was too symmetric and the camera might be confusing the top and the bottom. However even a strongly asymmetric image gave me similar results.
  • Swapping markers on my physical object.
    • That didn't do much, as if it is specific to the image I'm using.
  • Using different camera mode.
    • Tried camera-centring vs target-centric, but I've experienced similar results.
  • Occluding the other parts on the physical object.
    • I tried hiding others markers with sheets of paper to see if once isolated the problem would vanish. Unfortunately that didn't help.
  • Changing the order of parts in the dataset xml file
    • That didn't do much.
  • Removing parts from the multi target definition in the xml file
    • That didn't do much.
  • Embedding the generated multi-target and using that instead of using CreateDataset/Load/ActivateDataset/...
    • That didn't do much.
  • Tried all of the above both on an Android device and in the Editor
    • Same results.

Manual workaround

I've double checked in Unity that my multi target was setup properly by looking at in in the scene view. As you can see in  [multi-target.png] everything looks fine.

However at runtime I experienced the issue with the tarmac image target. As you can see in [tarmac-flip.png] it seems like the tracker thinks the marker is flipped by a 180 degrees both on the horizontal and vertical axis of the tarmac image.

[tarmac-flip.png]

The workaround I found, was to change the rotation of tarmac part (in the xml) so it has the rotation perceived by the tracker.

Doing so does fix the problem at run-time, however when you look at the layout generated by the MultiTargetBehaviour it now looks broken: tarmac is now on the wrong face [workaround-face.png, workaround-back.png].

[workaround-front.png][workaround-back.png]

However that's very custom fix, unless there is a way for me to detect those bogus markers when generating the MultiTarget.

Here is an example of this manual fix, the line in comment was the calibration generated. The line under is the one with the manual fix.

<?xml version="1.0" encoding="UTF-8"?>
<QCARConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="qcar_config.xsd">
  <Tracking>
    <ImageTarget name="tarmac" size="0.288000 0.201600" />
    <ImageTarget name="portrait" size="0.192000 0.288000" />
    <ImageTarget name="carpet" size="0.288500 0.192256" />
    <ImageTarget name="poster" size="0.192500 0.287564" />
    <MultiTarget name="MultiTarget_TaiwanCardboard">
      <Part name="poster" translation="0 0 0" rotation="AD: -1 0 0 0" />
      <Part name="carpet" translation="0.00396097 -0.265921 0.005339052" rotation="AD: -0.8884737 0.4441642 -0.1167058 1.761423" />
      <!-- <Part name="tarmac" translation="-0.0002542313 0.1524896 0.4015127" rotation="AD: -0.01696756 0.7283373 0.6850087 179.1176" /> -->
      <Part name="tarmac" translation="-0.0002542313 0.1524896 0.4015127" rotation="AD: -0.01696756 -0.7283373 0.6850087 179.1176" />
      <Part name="portrait" translation="-0.001491979 0.1536206 0.1490362" rotation="AD: 0.9999707 0.006713306 0.003720322 87.3789" />
    </MultiTarget>
  </Tracking>
</QCARConfig>

Generation of multi-targets

Since I wrote this code by trial and error, here might be the cause of the issue I'm experiencing. However I doubt it since  in order to get the MultiTargetBehaviour to generate the a layout that correspond to the layout of transforms I have in Unity.

Here is the code that generate the parts:

            foreach (var kv in calibrations)
            {
                var trackable = kv.Key;
                var calibration = kv.Value;

                var part = document.CreateElement("Part");
                part.SetAttribute("name", trackable.Name);

                var translation = calibration.position;
                part.SetAttribute("translation", string.Format("{0} {1} {2}", translation.x, translation.z, translation.y));

                var rotation = calibration.rotation;
                Vector3 axis;
                float angle;
                rotation.ToAngleAxis(out angle, out axis);
                part.SetAttribute("rotation", string.Format("AD: {0} {1} {2} {3}", -axis.x, -axis.z, axis.y, angle));

                multiTargetElement.AppendChild(part);
            }

As you can see there are some shenanegans I'm doing for generating a translation and a rotation that makes the MultiTargetBehaviour generate a proper layout. I wish there was a proper way to generate those based of unity transforms but eh!

Help!?

Unfortunately I'm running out of idea regarding what might be causing this. I haven't found any other fix beside that manual fix and unless I can find a way to detect when this manual fix is necessary I don't see an obvious way to fix it.

Let me know if you think of something.

Hopefully it's something obvious I'm not doing and not a bug :)

Thanks in advance for your help folks!

Note: I've attached the calibrated dataset.

-Sam

AttachmentSize
Package icon Archive.zip353.69 KB

MultiTarget trackable flipping orientation for a specific image target

December 26, 2017 - 1:42am #5

Hi Scadieux,

Though it has been a long time, but I wish you finished your demo.

I have been trying to use multi target(I want to track all the 20 sides of a icosahedron) as well and it has been a painful experience. The documentation is not clear and specific at all. And what you see in the scene and what you see in the game is totally different, which is very confusing.

For example, I modified my xml in a way so that in the scene of the Unity editor, the image target is perfectly aligned with my final object. But when I hit play, they are not.

After so many trail and errors, I finally concluded that the above error would appear when you are using a complicated rotation, for example:

<Part name="examplemarker" translation="0 0 0" rotation="AD: 0.3911556 0.2607704 -0.8826076 200"/>

If you just rotate along the x axis(or y, or z), it would work just fine, for example:

<Part name="examplemarker" translation="0 0 0" rotation="AD: 0 1 0 153.645645"/>

@Developers in Vuforia, you can recreate this and visualize the error easily.

Normally, vuforia would render the marker as a mask, which means that the virtual object behind the marker cannot be seen. Normally, the mask would align with the actual marker in the camera video view perfectly, but when a complicated rotation is used, the mask is not aligned with the actual marker in the camera!!

I really hope Vuforia can fix this or state this in the official documentation.

 

 

 

MultiTarget trackable flipping orientation for a specific image target

February 19, 2017 - 10:21pm #4

I'm trying to finish a demo by the end of the month, any help would be appreciated. Thanks!

MultiTarget trackable flipping orientation for a specific image target

February 15, 2017 - 11:29am #3

Something else I tried but without success is to use the different representations that the XSD allows for the rotation:

  • using radian instead of angles (AR)
  • using quaternions even tho it's not supported by the MultiTargetBehaviour (VuforiaUtilities will print out an error). It seems like it work just fine at run-time.
    • part.SetAttribute("rotation", string.Format("{0} {1} {2} {3}", -rotation.x, -rotation.z, rotation.y,  rotation.w));

But no luck, the same issue is happening :(

MultiTarget trackable flipping orientation for a specific image target

February 15, 2017 - 10:23am #2

While trying to debug this further I found an alternate case where I've noticed the same issue.

It seems like if I rotate the carper image target by ~120 degrees and tilt it it's enough to trigger the issue.

In the second image the blue dots are the origin of the image targets.

 

The MultiTarget produced by the calibration described in the previous post yield the expected layout when loaded in a MultiTargetBehaviour.

 

So far so good everything match. However at runtime I get this:

 

By trial and error I start playing with the sign of the rotation components of the carpet in the MultiTarget until I got it working at runtime. Here is the xml (the line in comment is the original one produced by the calibration while the next one is the working one):

<?xml version="1.0" encoding="UTF-8"?>
<QCARConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="qcar_config.xsd">
  <Tracking>
    <ImageTarget name="portrait" size="0.192000 0.288000" />
    <ImageTarget name="carpet" size="0.288500 0.192256" />
    <ImageTarget name="tarmac" size="0.288000 0.201600" />
    <ImageTarget name="poster" size="0.192500 0.287564" />
    <MultiTarget name="MultiTarget_TaiwanCardboard">
      <Part name="poster" translation="0 0 0" rotation="AD: -1 0 0 0" />
      <!-- <Part name="carpet" translation="-0.0361957 -0.2437551 0.04887364" rotation="AD: -0.1747361 0.2185272 0.9600589 118.5527" /> -->
      <Part name="carpet" translation="-0.0361957 -0.2437551 0.04887364" rotation="AD: 0.1747361 -0.2185272 0.9600589 -118.5527" />
      <Part name="portrait" translation="-0.003861563 0.1570481 0.1437356" rotation="AD: 0.9999988 0.001305654 -0.0008511422 87.63136" />
      <Part name="tarmac" translation="0.0006174204 0.1560513 0.3904745" rotation="AD: -0.02110673 0.73939 0.6729466 180.6585" />
    </MultiTarget>
  </Tracking>
</QCARConfig>

 

With this MultiTarget definition I finally got the expected result at runtime:

 

However the MultiTargetBehaviour now layout the image target wrongly:

 

I hope this second case of the issue will help you guys to find what I'm doing wrong.

Cheers,

-Sam

Log in or register to post comments