Log in or register to post comments

Using Advanced Camera API on HoloLens

August 23, 2019 - 6:21am #1

Hello,

 

I am working on an application using Vuforia on the HoloLens, and we would like to have some control over the image/video being analyzed by Vuforia. I saw in the FAQ that it is not possible to provide Vuforia a custom video feed, but I also saw that Vuforia offers the Advanced Camera API (https://library.vuforia.com/content/vuforia-library/en/articles/Solution/Advanced-Camera-API.html), which might be great for us. We could tweak some parameters of the camera and hopefully improve tracking in our case.

However, it seems to me that this API is only available on Android?

If so, is there any way to modify the camera footage (e.g. change contrast or brightness or even apply some image processing) on the HoloLens? Before Vuforia starts working on the footage, of course, so that hopefully we can have better tracking.

The license we have is not a problem.

 

Thanks :)

Using Advanced Camera API on HoloLens

August 30, 2019 - 5:21am #8

So unfortunately, it seems unlikely that we can influence the camera settings for Vuforia. And I am aware that Vuforia can not be fed custom images/video source (which on HoloLens might incur a significant overhead anyway).

 

Thank you for the information :)

Using Advanced Camera API on HoloLens

August 30, 2019 - 1:38am #7

Hi

I've discussed the issue with the devs internally. As you mentioned the VideoDeviceController provides in theory the needed flexibility to adjust the video input, however HL1 doesn't allow for much camera control and we don't know if the setups works correctly. In addition we didn't test for this specific scenario.

Unfortunately it seems that Vuforia's API does not provide this capability for UWP devices.

Thank you.

Vuforia Engine Support

 

Using Advanced Camera API on HoloLens

August 29, 2019 - 4:09am #6

I am sorry.

We are using MT360 for now with Vuforia 8.3.8 and Unity 2018.4.2. We have a Pro license.

Most objects we need to track are quite dark and have no specific texture (they are just a single dark color), unfortunately, which we cannot change.

I don't think the camera resolution is a problem. We tested Vuforia MT360 with the same objects on a tablet. The camera is apparently using 1280x720 on the tablet, and once we shine a strong light on the part it is recognized quite quickly.

Unfortunately, continuously shining a bright light is far from an ideal solution so we would like to change the HoloLens' camera settings. Or the image itself, if this is the only possibility.

I can query and change the Windows "VideoDeviceController", obtained from a MEdiaCapture object, but it does not seem to affect the image I see from Vuforia's CameraDevice.Instance.GetCameraImage().

Using Advanced Camera API on HoloLens

August 29, 2019 - 3:46am #5

Hi,

Ok, so the video quality is not as expected and you would like to improve it by tweaking the video feed.

One thing that is missing and I would be good to understand, is what features from Vuforia are you using? my assumption from the last post is image targets.

When using the feature, what issues do you see?

Thank you.

Vuforia Engine Support

Using Advanced Camera API on HoloLens

August 27, 2019 - 7:30am #4

Hi!

 

Simultaneous posting it seems :).

 

The issue we are seeing is that, when we look at the image (taken through CameraDevice.Instance.GetCameraImage(...)), it appears quite poor compared to what we see with our eyes.

The problem is, I think, that our objects are very dark. To the naked eye, the edges are easy to see, but on camera the object quickly becomes a dark blob/silhouette. So we want to try playing with the camera settings to hopefully improve the image and make the edges more visible/bright.

Using Advanced Camera API on HoloLens

August 27, 2019 - 7:26am #3

Update:

 

I found some other posts where it was said that the Advanced Camera API is only available for Android.

 

I have instead been trying to use the HoloLens' "VideoDeviceController" class.

However, I am not sure if this affects the global settings or not.

I create a Windows.Media.Capture.MediaCapture object, just to gain access to a VideoDeviceController, which I hoped is a global control and affects also the camera image for Vuforia. However, I see little difference when I change settings such as brightness, contrast, and exposure. I check this by occasionally taking the CameraDevice.Instance.GetCameraImage(...) and showing it in the world simply so I can see what it looks like (just for testing).

 

My question:

Is Vuforia overriding the camera settings?

Or is the VideoDeviceController only used by the specific capture object? (And in my case, my MediaCapture object's VideoDeviceController does not affect Vuforia's)

In that case, is there a way to get access to the VideoDeviceController Vuforia uses?

 

Thanks :)

Using Advanced Camera API on HoloLens

August 27, 2019 - 7:25am #2

Hi,

Yes, the Advanced APi is only available for Android.

In regards to your second questions, usually we do not encourage developer to "play" with the video being send to Vuforia as this usually results in tracking issues.

Could you please provide more information on why you want to tweak the video signal?( you mention to improve tracking :) however I am looking for more information to understand the use-case)

Thank you.

Vuforia Engine Support

 

Log in or register to post comments