Log in or register to post comments

URGENT: problems with iPhoneUtils.PlayMovie in combination with QCAR

January 19, 2012 - 6:29am #1

Hi,

We are working on a project for a high-profile client, and we need to show full-screen videos that are started from the AR view. We are running into several problems however, and hope you can help!

iOS
On iOS, general operation of the movieplayer is very good: it opens and closes quickly and without problems. I do set the ARCamera.active to false before starting the movie and back to true after it (it is my experience that if I only disable the QCARBahaviour component, it comes on automatically when control returns to the AR view, and that's not what I want in this case).

However, when the user closes the movie while it is playing by pressing the hardware-button on the front-panel of the iOS-device, and then re-opens the app from the home-screen, Unity continues but the tracking does not!

Android
When I make a benchmark Unity scene (without QCAR) and call iPhoneUtils.PlayMovie using a couple of GUI.Buttons, the behaviour is good: the movie starts and closes promptly.

When I make the same benchmark scene but in a project where the QCARUnityPlayer is used, the application hangs after the movie closes. This is of course an unusual situation (using the QCARUnityPlayer without using QCAR), but maybe it helps in troubleshooting the problem.

When I try to use iPhoneUtils.PlayMovie while the tracking is active, starting the movie is very slow, and closing it + restarting AR is too.

I'm trying to add additional states to my controlling object in the application, so that I can show a 'Starting video' and 'Starting AR' screen while the transition is in progress. But this turns out to be tricky:

  1. if I set ARCamera.active to false before going to the movie, the application hangs when coming back from the movie
  2. if I only disable the QCARBehaviour component before going to the movie, the tracking is automatically resumed when we come back into the AR view. This takes a lot of time, and this means I don't get a chance to present the user with a 'Starting AR' screen.
  3. on some devices, the OS decides that the application has become unresponsive when returning to the AR view, and the user is given a choice between closing and waiting. When Wait is selected, the application resumes normally, but many people will not try this. This happens on my LG Optimus 2X speed, but this may be an exception since very often the camera takes a unsually long time to close. The message "State change started: OMX.Nvidia.jpeg.encoder from 2 to 1" is then shown about a hundred times in LogCat.

So my questions are:

  1. is there something I can do to have tracking continue on iOS, after the user closes a movie using the front-panel button?
  2. what is the best way to stop and resume the tracking on Android, so that I (a) get a chance to show some screens to the user about what is happening and (b) the prevent the OS from deciding that the app is not responsive?

The behaviour is the same with PlayMovie as with PlayMovieURL by the way.

Edit: We're using QCAR 1.5.3 beta on Android, 1.5.4 beta on iOS, and Unity 3.4.2f3.

Thanks a lot in advance!
Lex

Re: URGENT: problems with iPhoneUtils.PlayMovie in combination w

March 8, 2012 - 4:26am #4

Hi,

Here's a short writeup of our solution to the problem. It turns out the problem wasn't with QCAR after all.

We found that the reason some Android/AR apps restart slowly after playing a movie is that on Android 2.x the GPU memory is cleared whenever the current activity goes to the background. That happens at all kinds of occasions: when you receive a call, go to the home screen, and also when starting the videoplayer, which is also an Android activity. Also see this post on StackOverflow for this.

Since API level 11 (that's Android 3.0, i.e. Honeycomb, for tablets), this function can be called:
[CODE" />public void setPreserveEGLContextOnPause (boolean preserveOnPause) Since: API Level 11[/CODE" />
But that doesn't help us here.

In both our applications we had pretty large textures, some custom shaders and in one of them also a moderately complex model, so reloading all this to the GPU took some time.

Ultimately our solution was to never leave the 3D scene to show the video, but instead use the Mobile Movie Texture plugin from Defiant Development, and start an audio stream in parallel (and hope for the best regarding synchronization).
We did have to limit the framerate of the video to about 18 fps, since the plugin couldn't (maybe still can't) skip frames, and we wanted to prevent the video from lagging behind the audio in case the decoding couldn't keep up on slower devices.

In the mean time we have also deployed the plugin on an iOS project, and it's working very well.

In general I really recommend to NOT use Unity's VideoPlayer in combination with QCAR.. It has given us a lot of trouble.. For Unity3D 3.4 and QCAR 1.5.6 beta we had code up and running to manage the combination with the tracker. I.e.:
- pre-emptively stop the tracking before starting the video, using a modified QCARBehaviour script
- and start it manually after we had returned from the video, half a second little later than directly in OnApplicationPause(false), because otherwise on iOS the app would often hang, especially with movies longer than 30 seconds.
But this seems to have broken with Unity 3.5 and QCAR 1.5.9/10.

If anybody has gotten this to work properly on both Android and iOS, we'd love to hear about it!
In the mean time, I recommend you check out the Mobile Movie Texture..

Lex van der Sluijs

twnkls | augmented reality
www.twnkls.com

Re: URGENT: problems with iPhoneUtils.PlayMovie in combination w

January 20, 2012 - 7:09am #3

Hi Kim,

thanks for the fast response.

Good to know that it's not anything we did wrong regarding the resuming of tracking after closing the app during movie playback.

One to two seconds to go back to AR from video sounds really good. On which device(s) did you get that result?

I think we only have that performance on our Samsung Galaxy Tab 10.1 and Galaxy SII phone. On HTC Desire Z, LG Optimus 2X, or Galaxy Tab 7", it often takes 10 seconds or more..

By the way, regarding the statement I made below "When I make the same benchmark scene but in a project where the QCARUnityPlayer is used, the application hangs after the movie closes. This is of course an unusual situation (using the QCARUnityPlayer without using QCAR), but maybe it helps in troubleshooting the problem.", I just realized that this may actually become a not so unusual situation for us: we are about to start work on a larger application with multiple scenes. Some if these scenes have AR, others don't. So it would be best if the player in the android project can also work with scenes where there is no ARCamera present... But I digress..

In one project, the videos are 320x480, in MP4, H.264 format with AAC audio, about 2 to 3 MB in size, compressed using Quicktime Pro. In the other one the size varies between 2 and 10 MB (same resolution and compression, Format Factory was used for the compression).

I'll send an e-mail to support with APKs from two projects that we're working on, that exhibit the behaviour I described.

Thanks again,
Lex

Re: URGENT: problems with iPhoneUtils.PlayMovie in combination w

January 19, 2012 - 8:56pm #2

Thanks for the report. I can confirm the iOS issue with restarting the app after closing it during video during playback. We'll look into this.

I'm not able to reproduce the performance issues on Android, however. I'm starting with the ImageTargets sample and playing an 8MB .mp4 video when a target comes into view. The video starts immediately, and when I press the back button it takes about 1-2 seconds to jump back to the AR camera view.

What video format are you using, and how large is the video?

Could you send an apk of your app to

?

- Kim

Log in or register to post comments