Log in or register to post comments

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

March 17, 2020 - 6:12pm #1

Hiya!

We're working on upgrading a complex, interactive HoloLens (vanilla) project to HoloLens 2, and as part of the upgrade, our client has requested image tracking support be added to the application.

For background information, we're very experienced both with HoloLens development and with implementing Vuforia-based image tracking into projects. We have successfully created several multi-platform projects which use Vuforia-based image tracking, including one in HoloLens that uses a version of the "Vuforia HoloLens Sample" project available on the Unity Asset Store in early 2018.

As an organisation, we specialise in inventing novel ways of conveying a feeling to an audience, so we are well-accustomed to working with experimental technologies, and we feel that this is a challenge that can be solved.



Our first approach was to attempt to port the project into a framework created using the "Vuforia HoloLens 1+2 Sample" project available on the Unity Asset store, however we found that this sample project is not capable of supporting our client's project, for the following reasons:

  • It is not based on any version of the HoloLens MRTK v2.x plugin provided by Microsoft, therefore does not include many of the key features we want to leverage from the MRTK v2 for HoloLens 2 development, essentially tying us to an already deceased development pathway.
  • It is based on a rudimentary version of the HoloToolKit which does not contain several key features available in the HoloToolKit as of early 2018, such as various interaction, spatialisation & networking functionality, each of which is required by our project.
  • It is sufficiently distinct from later 2018+ versions of the HoloToolKit (and even more distinct from the current MRTK v2.3) that no clear upgrade path exists to implement image target tracking into our existing HoloLens project using the sample project, or into a HoloLens 2 variant of the project made using MRTK v2.x.
  • It does not allow us to use our own, fully featured & heavily customised version of the HoloToolKit, nor does it allow us to use our preferred version of the MRTK v2.x.

Our experience of the Vuforia HoloLens sample seems to echo the general consensus of the reviews for that asset - that it is not fit for purpose when working with HoloLens 2.



As an alternative approach, we spent quite some time trying to Frankenstein some version of Vuforia with versions of the MRTK as well as with our customised version of the HoloToolKit, but we have been unable to get any of these proposed frameworks into a workable condition.

The most common issue we encountered during these experiments was that Vuforia simply would not work. Instead it would silently fail, despite using identical code that worked as expected on other platforms such as iOS and Android, using the same project. In two of these attempts we were able to get Vuforia working on the HoloLens using our customised version of the HoloToolKit, however the HMD lost all ability to orient itself within its virtual space as soon as our test image target was detected, making the framework entirely unusable.

As yet we have been unable to get Vuforia working at all on the HoloLens 2 using the MRTK versions 2.2 & 2.3, using a more conventional approach of importing Vuforia via Unity's Package Manager or from an Asset Store package, and then adding VuforiaBehaviour + DefaultInitializationErrorHandler components to the Main camera, and placing an image target in the active scene.



What we need:

Ideally, we need to be able to use an encapsulated version of Vuforia - as one would with a Vuforia-enabled project on any other platform, so that we can use whichever version of the MRTK or HoloToolKit best suits the needs of the project, and upgrade that module as needed, completely independently from the Vuforia module. Every other plugin we're using allow this form of encapsulation, and we see no reason why this should not also be true for a Vuforia plugin as it merely requires a few additional components placed onto the Main Camera, and an image target.



So we have two questions:

1) For PTC/Vuforia: Is there an existing timeline for an official release of a functional Vuforia plugin for Unity-based HoloLens 2 development which can be used in conjunction with independently-sourced versions of the MRTK v2.x released by Microsoft? If so, what kind of release date should we expect for this plugin?

2) For other developers using the MRTK v2.x and HoloLens 2: Has anyone else managed to get Vuforia working with any version of the MRTK v2? If so, any insight you can provide would be invaluable!

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

May 19, 2020 - 4:49am #8

Yes I'm using simple image trackers in unity 2019.3.7.

The trackers are found and tracked. They are just placed at a wrong position (too much positional offset away from the camera according to my debugging output).

I've tried different approaches, different projects, including a clean project from the HL1+2 sample from the store.

My device is not broken because my older vuforia projects work just fine.

It's taking up a lot of my time.

I was thinking about trying the samples in older unity versions but according to the comments in the unity store that would also be a waste of time.

I'm afraid we just have to stick to old working codes (holotoolkit and legacy xr plugins) in old unity versions. 

A bit sad considering I'm looking to find a proper image tracking + model tracking solution and I thought Vuforia would be a good fit.

Considering the price tag for model based tracking I would expect it to work better.

 

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

May 15, 2020 - 4:41am #7

Hi,

Just to clarify, we do support MRTK v2.x and it is avaiable in our HL1+2 sample app available through the Unity store.

Could you please elaborate on discovered completely wrong? Are you referring to Image Targets?

Thank you.

Vuforia Engine Support

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

May 14, 2020 - 9:47am #6

 

Hi Vuforia,

I have the same issue. Also works in the editor using the webcam.

On the hololens my object gets discovered but then positioned completely wrong.

I'm on Unity 2019.3 working with the new 'XR Management' & 'Windows XR Plugin' package.

I assume Vuforia is not supporting it yet? Any plans when this will be implemented so I can wait for that release?

Is there a way to fix / hack this for now? Are we forced to stick with the legacy methods & packages ('Windows Mixed Reality')?

 

Kind regards,

Mathieu

 

 

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

May 6, 2020 - 6:35am #5

Hi,

Microsoft recommends Unity 2018.4 for Hololens1/2.

Also please build using ARM64.

I would recommend to install our Hololens Sample app and follow this article: https://library.vuforia.com/content/vuforia-library/en/articles/Solution/Working-with-the-HoloLens-sample-in-Unity.html

Thank you.

Vuforia Engine Support

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

April 30, 2020 - 10:11pm #4
  1. I started a vanilla unity project 2019.3.6f1
  2. Added MRTK (just the foundation unitypackage for version 2.3.0)
  3. Switched platform to UWP
  4. Performed the basic configuration steps
  5. Enabled VR and set depth buffer to 16bit
  6. Installed Vuforia 9.0.1.2 Core from the unitypackage available off the download section of Vuforia's site.
  7. As per instructions many have posted, no need to create an "AR Camera" game object using the Vuforia menu, simply added the following to the MRTK default main camera: Vuforia Behavior, Default Error Initialization Handler.
  8. Added an ImageTarget based on the astronaut sample imagetarget and placed some basic content underneath it (Made sure to use unlit textures so there are no issues with single pass instanced rendering)
  9. Added the defaulttrackableeventhandler and turnoffbehavior script to the IMagetarget
  10. Test with webcam in player: properly detects the target and displays my content
  11. Created build for ARM  and deployed to hololens 2
  12. No go, absolutely nothing happens

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

April 6, 2020 - 5:12pm #3

did you end up getting a vuforia app to work on your hololens 2? my app works in the unity editor playmode but when i deploy it to the hololens 2, it doesn't track my image.

Lack of decoupled Vuforia support for MRTK v2.x and HoloLens 2

March 18, 2020 - 2:12am #2

Hi,

I want to confirm that we use MRTK v2.2 in the Vuforia Hololens 1+2 samples.

We use the MRTK assets and we do have in every scene MRTK, in addition in our library we have following article describing how to work with the HoloLens sample project.

https://library.vuforia.com/content/vuforia-library/en/articles/Solution/Working-with-the-HoloLens-sample-in-Unity.html

Please let me know what issue you encountered?

Thank you.

Vuforia Engine Support

 

Log in or register to post comments