Log in or register to post comments

Sometimes vuforia just doesn't start

April 19, 2014 - 5:16am #1

So I have an app with several scenes that coorspond to books (but that doesn't matter). Each scene has the same setup, ARCamera, various ImageTargets, etc. 

One of my features that exposes this issue relatively often is taking screenshots. When the screenshot is performed, I switch to another UI screen, which displays the screenshot you just took, lets you annotate, send to social networks, whatever. Anyway I found that it was a good user experience to stop the AR while this screen was up, otherwise you'd hear whatever random activitiy going on behind the screen depending on what page they took a screenshot of. So I disable vuforia during this time. When the user closes the screenshot display UI screen, I re-enable vuforia. Most of the time it works, but about 10% of the time it doesn't come back. It's just black. 

The user can fix it themselves, by opening the book selection and just re-loading that scene, but obviously that's not right. So I was looking for a way to detect the problem has occured and then reload it myself or kick it on the butt to try and re-init. But i haven't found a good way to do the first part yet...detecting that vuforia is having some issue.

I found this: https://developer.vuforia.com/forum/unity-3-extension-technical-discussion/how-does-oninitialized-work

I had planned to just watch and see if OnInitialize gets called....if not after some amount of time...then do something about it. But, it only gets called once, ever. If you disable and enable QCARBehavior...the subsequent times, even when it does get initialized, that event never happens. 

I guess it's because I'm not properly disabling QCAR. 

So...

How to properly shut down the QCAR stuff, and safely (without fail) enable it again? Right now I am just doing this...

QCAR = go.GetComponent<QCARBehaviour>();

QCAR.enabled = false; and later doing QCAR.enabled = true;

(fails some percentage of the time).

Alternative question....how to verify and insure that vuforia runs. This was just one example of it failing to start...there are others. About 10% of the time of a fresh launch of the app, the first scene to load will fail (vuforia, black screen, everything else works, UI, etc) to start. To fix, just do it again until it does start.

This is basically the only showstopper remaining for this app.  

 

 

Sometimes vuforia just doesn't start

May 29, 2014 - 1:39am #30

Nice. I look forward to trying out whatever other goodies it contains.

Sometimes vuforia just doesn't start

May 29, 2014 - 12:14am #29

UPDATE:

the issue is fixed in the next upcoming release of Vuforia (no official dates for it, but expected soon).

 

Sometimes vuforia just doesn't start

April 28, 2014 - 6:05am #28

Yes, sure, you just need to check QCAR is not null before accessing QCAR itself (e.g. calling QCAR.enabled). Sorry i did not include such safety check.

 

Sometimes vuforia just doesn't start

April 28, 2014 - 5:45am #27

This seemed so harmless at first. Now I see that OnApplicationPause runs before Start...so...qcar wasn't defined yet. Already fixed and trying tests again.

Sometimes vuforia just doesn't start

April 28, 2014 - 5:35am #26
NullReferenceException: Object reference not set to an instance of an object
MenuController.OnApplicationPause (Boolean paused) (at Assets/MenuController.cs:23)
 
This seemed like it more or less was working, but then during some unrelated tests I got a crash in iOS. This surprised me because it never crashes. So I went back and looked...noticed the error up there. To be sure, I just downloaded your actual file and it definitely has a null ref. 

Sometimes vuforia just doesn't start

April 28, 2014 - 12:40am #25

For your second question about the "rare black screen on fresh loads" issue,

it would be great if you could create a new topic / thread; not because it is unrelated to the current one, but since it is a slightly different issue, I would prefer to have a dedicated (and separate) thread (if you like, you can mention a link to this thread as it is partially / potentially related issue).

Thanks.

Sometimes vuforia just doesn't start

April 27, 2014 - 11:30pm #24

Great.

One last question...is there a way to detect that QCAR has gotten itself into a simularly weird state? We do sometimes see, very rarely, the black screen on fresh loads. It's so rare that it's been impossible for me to figure out what leads to it...just every once in a while I'll see one happen. When it does happen the behavior is simply that I launch the app, straight away load an AR scene, and see black. Relaunch it and find no problem.

In earlier builds of Vuforia (or could be earlier builds of Unity) I would see it about once a day during development, but lately it's very rare.

If there is no such technique to detect this, I would strongly suggest implementing something that can return such an information.

Sometimes vuforia just doesn't start

April 27, 2014 - 11:35am #23

This works in the simple sample. And it works in my particular production. And it doesn't mess up anything in iOS that I can see so far.

Happy to hear your confirmation that this workaround works for you (although I understand that you will want to wait for some more device testing to be sure). 

 

Your more recent one makes me less nervous because I understand what it's doing.

Indeed, the OnApplicationPause() technique is basically doing a very simple thing: making sure that Vuforia (camera, trackers, etc) gets explicitely stopped/deinited when the application is resumed after quiting the URL Web Page and returning to your scene with the Menu ON (see the "bool resumed = !paused;" line of code),  consistently with the fact the QCARBehaviour component was not enabled.

It is basically a way of manually enforcing that the internal Vuforia state and the QCARBehaviour.enable state are still maintained 'aligned', when exiting the WebPage and returning to the Menu scene.

And this is acting on the actual cause of your problem (as the misalignment was then causing the black camera issue when exiting the Menu to return to AR mode, for instance).

So, you can of course run a few more tests on other devices as applicable, if you wish so, but you can rest assured that we have also tested this quite extensively and we have discussed this technique with our engineering team, and it seems to be a reliable and technically correct approach. 

 

 

 

Sometimes vuforia just doesn't start

April 26, 2014 - 10:59am #22

To be clear, what has been done here is...

Continue using the enable/disable technique to control QCAR. And then adding the OnApplicationPause code. The other stuff that was part of the older work around was removed. 

Result:

This works in the simple sample. And it works in my particular production. And it doesn't mess up anything in iOS that I can see so far. Of course this is just on my personal devices. Real testing is costly so I still need someone from the team to look at it.

I also had working code in production by combining part of your previous stuff with CameraDevice's init() and deinit() methods. Your more recent one makes me less nervous because I understand what it's doing (I don't know the proper care and feeding of deinit). But either work around is fine once someone from the team says it's a sound idea. So now there are 2, appear to be working, work arounds. That should be plenty of hint for the developer to actually solve it for good. 

Sometimes vuforia just doesn't start

April 25, 2014 - 9:07am #21

Attached source file of modified script.

AttachmentSize
Package icon MenuControllerModified.zip768 bytes

Sometimes vuforia just doesn't start

April 25, 2014 - 9:00am #20

Update:

the Vuforia team has been informed of this, but I don't have feedback to share yet; 

in the meantime, as a short-term workaround, may I ask you to do one more test with this modified version of your MenuController.cs script ?

In the MenuController.cs that you originally attached to a previous post, could you add the following code:

void OnApplicationPause (bool paused) 
{
    Debug.Log (" Application was " + (paused ? "paused" : "resumed"));
    bool resumed = !paused;
    if (resumed && !QCAR.enabled) 
    {
        //Debug.Log ("----- Manually force QCAR stop, when the app is resumed but QCARBehaviour is disabled....");
        QCAR.enabled = true;//enable
        QCAR.enabled = false;//disable 
    }
}

And let me know ? (I'm referring here to the simple repro-case scenario with the MenuController.cs script attached to an empty gameObject, as per your description).

 

Sometimes vuforia just doesn't start

April 24, 2014 - 5:58am #19

Just an update to let you know I tried on the recently released 2.8.9 version...all same results. 

Sometimes vuforia just doesn't start

April 24, 2014 - 3:16am #18

Ok, thanks for your tests and the feedback, I am raising this with our team. Will let you know as soon as I get more feedback from the team.

 

 

Sometimes vuforia just doesn't start

April 24, 2014 - 3:11am #17

Ok. Here are the results...

The first time I tried your work-around code, I was just copying the new methods for Start and Stop and using that instead of the QCAR.enabled = false/true. That is the thing that was resulting in a halfway fix. 

Now that I am using all of your code, which first uses the QCAR.enabled = true/false and then calls a method to handle all the particulars...the new result is that doesn't work at all. Not even in the simple sample I built. It just behaves exactly as the bug already did. 

From reading old versions of vuforia, we can see that most of your code is actually being called already from those enable and disable bits (by reading the OnEnable and OnDisable of QCAR). So...i guess it's not surprising that something would get messed up...because now we're basically doing it twice.

I'm not asking for a feature, or an inconveinence to be addressed. We have a bug. It's repeatable. It's severe. You have a testcase that will always reproduce the bug. Please ask a developer to fix the bug or look at this thread.

As I said before, I was able to make a solution using deinit()...but I need a developer who works on the middleware to have a look to make sure it's safe and viable (but preferably to just fix the bug).

Sometimes vuforia just doesn't start

April 23, 2014 - 11:24pm #16

Oh...I think I missed something in your point that I understand now. You weren't replacing the QCAR.enabled = true technique...you were are adding to it. Duh. I will try that soon and report back. 

Sometimes vuforia just doesn't start

April 23, 2014 - 1:05pm #15

Hi, as I said in my previous post, the suggested way is to disable QCAR and to also stop the trackers and camera (see my code snippet); this implies that you can still rely on QCAR OnDisable / OnEnable methods, if your application needs so.

But I will also check with our team on whether there are additional tips to be aware of on this issue nd a deeper techical assessment on the internal behaviour of Vuforia in such a specific case.

 

 

Sometimes vuforia just doesn't start

April 23, 2014 - 7:26am #14

So with the correct combination of Deinit and Init (when you want to turn it back on) along with the start and stop...in my limited testing just now...it seems to work. It will take significant resources to test fully though. I really worry about it as a solution because it's obviously not the long term one.

Those methods in the old vuforia that handle OnEnable and OnDisable also have a bunch of other things that I don't quite understand. There's stuff in there about video modes, surface projections, orientation. 

The point is...this solution we are cobbling together is basically rewriting what is already in QCAR if we could read it. The danger is that these methods (StartQCAR and StopQCAR) have been updated from the old versions I'm reading...to handle some corner case that I won't find in time...meanwhile if we had simply fixed the bug...or post the updated methods so we could know with confidence what to copy...that would be better.

Sometimes vuforia just doesn't start

April 23, 2014 - 6:55am #13

Yea...i loaded up an older version of vuforia where you can still read the source...

QCARBehavior has an OnDisable...that's why disabling it works to do something. It calls something called StopQCAR which contains mostly the same code you had, but also has Deinit() and ClearVideoBackgroundConfig()...

One of my earlier questions was...has this internal source been updated. If it hasn't been updated then we can probably safely rely on it. Trying it now anyway.

Sometimes vuforia just doesn't start

April 23, 2014 - 6:51am #12

If I use QCAR.enabled = false I get a very different behavior. The augmentables are disabled as expected. My project is setup the way you would use vuforia with Unity. ImageTargets have children and the eventhandlers inform them to do stuff...ie like your samples.

I can't read the source for that QCAR file, but I imagine it has some code for OnDisable in there that probably is doing this stuff.

Sometimes vuforia just doesn't start

April 23, 2014 - 6:43am #11

It's very easy to build a work around, for that simple testcase. And of course your suggestion works

Glad to hear confirmation that it works for you too.

but I suspect the AR tracking and all of that is stopped, but that the objects that were augmentations are left afloat somewhere.

I am not fully clear on what you are saying here. If the Vuforia camera and trackers are stopped, nothing is really happening on the Vuforia side...the augmentations are not controlled by Vuforia; Vuforia simply delivers you a list of Trackable objects, each with with a Pose (position + orientation), on top of which you can attach your augmentations (as you are doing, I suppose). Contolling the augmentations (for example, removing/disabling them, or changing their position, or similar things)  is an application-logic matter.

In other words, whether you simply use the "QCAR.enabled = false" approach (your first approach), or you use the second approach by stopping the Trackers explicitly (which solves the black screen problem), you would still need to take care of managing your augmentations at app-logic.  Note that the Vuforia API offers various ways of helping you to handle the augmentations; the most common one is the use of the ITrackableEventHandler, but there are also other possibilities; see for instance this article which explains how you can query the StateManager to get the list of active trackables at any frame:

https://developer.vuforia.com/forum/faq/unity-how-do-i-get-list-active-trackables

 

Let me know if I miss something in my reasoning.

 

 

 

Sometimes vuforia just doesn't start

April 23, 2014 - 6:30am #10

The test project I built is designed to isolate the bug in vuforia. It's very easy to build a work around, for that simple testcase. And of course your suggestion works. However, in my production project it's much more complex and it doesn't work. I found that it worked in the editor, but on the device only half-worked. The working part is that I don't get a black screen when I run those tests. The not working part is that I could still "hear" stuff happening behind my menus. That tells me that some or all (probably all) of the augmentations are still running. It doesn't happen on the editor so I can't look to see what it's doing back there, but I suspect the AR tracking and all of that is stopped, but that the objects that were augmentations are left afloat somewhere.

I've spent a few hours now trying to figure out where it's breaking down, but so far haven't been able to. I also have this feeling like we're going down a strange road. We identified a bug that happens all of the time, performing a user operation that's very typical in AR apps. Most AR apps I've seen will have some kind of "go to our website" button...usually about getting an image target to print. The other common one, is using some kind of webform to interact with a social network...because you want to snap a picture of the AR and share it.

So...why not just fix the bug?

 

 

Sometimes vuforia just doesn't start

April 22, 2014 - 9:45am #9

I would say, let's go one step at a time;

please try the start/stop method and report back if you still face the issue; if that solves the issue for you (it seems to solve for me, but I did not test on many devices), we can discuss the next steps and your other questions, based on a solid baseline.

 

 

Sometimes vuforia just doesn't start

April 22, 2014 - 8:58am #8

I haven't found a device it doesn't happen on. We have about 100 devices we use for QA...but I didn't perform the test on all of them. After about 15 I stopped because 15 was already very bad and the 15 happened to be mostly samsung devices which occupies the majority of the Android market. 

I had noticed in my tinkering that the stop and start method seemed to work, but I had also found a thread that showed some other problems with those, and a recommendation to use Unity's enable/disable feature. So I was avoiding it. I'm not at the office right now, but tomorrow I'll hook it all up in the real app to use the start/stop way now and see how it goes.

I have 3 questions...

1: Should I continue using enable/disable for iOS or do you feel the start/stop way is superior?

2: Do you know of a way to detect the problem is happening? Of course for this particular thing, I will just use the start/stop methods, but occasionally (it improved a lot after 2.8.7) I still get a black screen on the first run of the app (the AR part I mean).If I had a bit of code to detect the issue, then I could just correct it (stop/start or disable/enable). Like I said before, I could do a screenshot, check the screen for blackness, and react, but that's going to ding everyone's framerate even if they aren't having the problem.

3: If the enable/disable way is best, but just has a bug right now, then when can I get a fixed build?

Sometimes vuforia just doesn't start

April 22, 2014 - 6:55am #7

Hi, I had a hard time to reproduce it on my device, but in a couple of cases I managed to see the issue... I guess on certain devices tis may occur more often than on others;

anyway, the issue seems in the fact that when coming back from the webview to the menu view, internally some Android onResume() call triggers some QCAR start process, despite the QCAR component being disabled.

So, when you then click on "Return to AR", occasionally this may mess up with the camera initialization;

so, I have tried this simple workaround of explicitly starting and stopping the Camera and the Trackers whenever you enable / disable the QCARBehaviour component, i.e. by slightly modifying your script code with this:

void invokeMenu()
	{
		Debug.Log ("Invoking Menu...");
		QCAR.enabled = false;
		StopTracking (); // ADDED THIS LINE
		appState = StateApp.Menu;
	}
	void returnToLevel()
	{
		Debug.Log ("Return to Level (AR)...");
		QCAR.enabled = true;
		StartTracking (); // ADDED THIS LINE
		appState = StateApp.LevelRunning;
	}

	private void StartTracking() {
		ImageTracker it = TrackerManager.Instance.GetTracker<ImageTracker>();
		MarkerTracker mt = TrackerManager.Instance.GetTracker<MarkerTracker>();
		CameraDevice.Instance.Start ();
		it.Start ();
		mt.Start ();
	}
	
	private void StopTracking() {
		ImageTracker it = TrackerManager.Instance.GetTracker<ImageTracker>();
		MarkerTracker mt = TrackerManager.Instance.GetTracker<MarkerTracker>();
		it.Stop ();
		mt.Stop ();
		CameraDevice.Instance.Stop ();
	}

 

Could you try this and see if it gets any better ?

 

Sometimes vuforia just doesn't start

April 22, 2014 - 4:49am #6

Hi,

thanks for the detailed report and the easy setup to help reproduce the issue... I will investigate this today and get back to you asap with some feedback.

 

Sometimes vuforia just doesn't start

April 20, 2014 - 12:46pm #5

I built a tiny project that exposes this for you.

You actually don't even need the project itself...you just need one file which I've attached to this thread.

Make a new project.

Switch platform to Android

Add vuforia 2.8.7

Add an ARCamera

Add your favorite imagetarget from target manager (you don't actually need this)

Add an ImageTarget (again, you don't need to do anything with it)

Add the MenuController.cs file to the project

Make an empty GameObject and attach the MenuController

Build and test it out.

Explanation: By default the level will launch in AR mode,  you will see the camera working, etc. Pressing the menu button will switch the app state into a menu state. The AR is disabled because the expectation is that some kind of menu is there. 3 buttons. You can return to the AR, which will enable the AR and switch the app state back. You can reload the level, which does what it says. You can load up a webpage, which will load this thread on your device. 

There are 2 ways to see the problem.

First: Launch the app, enter menu, load the webpage, press back (some devices need to press it more than once or have other ways of switching tasks), find the app where you left it (on the menu state), return to AR. It will enable AR and switch the app state, but something will fail and you'll have a black output. You can still use the menu button, if you toggle back and forth it will fix the issue.

Second: The same as the first, but when you go back from the webpage to the app, this time reload the level. One would really expect this to not fail because everything is getting reloaded, but it will fail. If you use the menu to reload it again, the second time it will not fail. You could also just toggle back and forth between the menu and the ar to fix it.

Basically, whenever the webpage is loaded...the next time qcar tries to start, it will fail. Subsequent times will be ok. 

AttachmentSize
Package icon MenuController.zip698 bytes

Sometimes vuforia just doesn't start

April 20, 2014 - 12:22pm #4
I have found an even easier way to exposing this that also shows up even on a fresh scene load.
 
launch app
 
load scene that does something with ar
 
use menu system. all this does is disables the qcarbehavior and then displays a menu (using ngui) of other scenes to load. It's a very typical UI built with NGUI. 
 
click a button that triggers a website to load (one of my help buttons). In Unity the code that is doing this is merely Application.OpenURL(someurl). 
 
The app will switch to the device's default browser and load a webpage.
 
Press the back button on the device, it will go back to our app, right where we left off.
 
Load a scene...broken AR.
 
From here I can use my debug buttons to disable and enable QCAR, to fix it. 
 
If instead of the above, if I first load up that help page, using Application.OpenURL(someurl). then back and load a scene...its all fine and works. But if I first load a scene with AR, then do the webpage stuff and back and load a new scene with ar....broken. 
 
This is very repeatable on several Android devices (iOS works). 

Sometimes vuforia just doesn't start

April 20, 2014 - 10:29am #3

More info...

I have tried a handful of other Android devices and they all do the same thing. 

iOS however, doesn't have the problem. 

Sometimes vuforia just doesn't start

April 19, 2014 - 10:58am #2

I have since found a reliable way of reproducing the issue. Make the app load up a webpage and then come back to the app. The way I'm doing this is something that my app must do sometimes anyway...I use a plugin called UniShare to send screenshots to social networks. If the user isn't authorized already, it will load a bit of html up and let the user login, then return to whatever you were trying to do (post a picture). It's pretty typical. 

I disable QCARBehavior before entering the UI with this functionality, and it gets enabled when this screen is left (back to AR stuff). If during this period of time, I need to access that login screen, that somehow puts me into some state where the QCAR isn't recoverable by enabling it. I can delete the app data from the device, so that it will always need to authorize...so I know this is leaving it in a weird state...it happens 100% of the time that I perform the test. 

I have put some floating buttons that are always up no matter what the app is doing, using OnGUI, one disables and one enables. When I find the app in this state, if I push enable...it won't fix. I imagine that internally this does nothing because the component is already enabled. If I press disable and then enable, it comes back. 

I can make a dirty hack to fix this at this point, but I'd still need a way to test that this is happening first. With the html bit, I know it will happen 100% of the time, but as I said before...occasionally it also happens in other places...for example on a fresh load of a scene and seldomly upon recovering from a text message.

I would love a solution like...

if (bQCARIsSickOrOtherwiseNotInitialized) { ReInitAll(); } 

I could make a really hacky way of doing this by grabbing the screen and testing for how much of it is black, but that seems pretty brutal if not a little funny...

Also BTW...in the previous version of vuforia we could read the source code in QCARBehavior and see what it's actually doing for Awake and OnEnable, that could give us a lot of hints, but newer versions don't allow this. It looks like everything is packed into a dll. If I override OnEnable to try and solve this, I will overwrite whatever you had in there. I can read the old versions (2.6.7) and copy that code in, but I don't know if they were updated.

My device is a XiaoMi M2, super popular phone in China.

Log in or register to post comments