"We offer new support options and therefor the forums are now in read-only mode! Please check out our Support Center for more information." - Vuforia Engine Team

Play remote video on image use calayer (sample code)

By TimLai on Sat, 07/07/2012 - 14:17

I would like to share my recent work about playing video on trackable image, and ask some help.

Thank for vuforia team to release the sample code about how to play video using the vuforia sdk, they use avassetreaderoutput to render video pixel on the opengl texture.

However avassetreader can only work on local video file, if you want to play remote video, you need use native mpmovieplayer and into the fullscreen mode.

I try to solve this problem.

The idea is simple.

1.I don't use avassetreader to read the video pixel, instead I use AVPlayer and AVPlayerLayer which can play remote video file.

2.I need to convert opengl modelview matrix to CATransform3D so the AVPlayerLayer will attach on the trackable image.Thanks for Hammer on stackoverflow, he shared an example about how to make this.

3.I tried to render camera background use opengl and AVPlayerLayer at same time, but the performace is not good enough, so I use another calayer to render the camera background. There is a bug in 1.5.8 when getting camera frame, thank for andersfrank the problem was solved here.

I put my code on pastebin:

http://pastebin.com/kmeVZ7jy

 

I can play remote video now, but still have some problem need to solve:

1. I can't make the video flip up-side down.

2. it's also strange that I need to set m22 and m33 negative in scalingMatrix or the convertion between the modelview matrix and catransform3d is not correct. 

3. When I rotate my iphone, the attitude of AVPlayerLayer is not quiet well, may be the convertion is not good enough.

 

Image removed. 

this one looks good.

Image removed.

this one is not.

 

I was stuck on this problem about 3 weeks. Please check the code and give me some feedback.

Thank you very much.

Good news here, I just fixed all the problem, for question 1 and 2, there are some column-major and row-major matrix convertion problem from GLKMultiply, I just replace all of them by ShaderUnits::multiplyMatrix. For question 3, you can fix it by set

junkieHead

Fri, 07/13/2012 - 06:06

Permalink

Dear Tim,

I tried to integrate your code in Image Target Sample App of SDK but it is rendering errors such as QcarUtils.h not found.

I ahve also checked for the header paths and also for the frameworks included such as quartzcore but still unable to run the code.

Dear junkieHead,

To make my code work, please follow the steps below:

1.Open the "ImageTragets" sample code project. (Maybe you should backup the original version first.)

2.In the project, create new class called ARCALView, the copy and paste my code to finish this class.

junkieHead

Thu, 07/19/2012 - 09:59

Permalink

Hi Tim,

Solved most of the issues, the one which i am left with is that i need to call a UIView nib file after i tap on the custom CALayer which i have overlayes.

I am using tapgesture recognizer in which i am trying to use the following:

 

tgotzsche

Tue, 08/07/2012 - 19:03

Permalink

it's solved.

Had to do with ithat I changed code to call initVideo after target was identified rather than initially. I split initialization and loading into 2 methods and called them at proper time, then BAD_ACCESS problem is gone.

junkieHead

Fri, 08/10/2012 - 08:41

Permalink

Hi Tim,

I am facing some issues with the video play code.

Its working flawlessly when being tested on iPhone4 but when same code is tested on iPhone 4S, its through exec_bad exception after few seconds of tracking.

Do you have any idea why is this happening?

tgotzsche

Fri, 08/10/2012 - 17:46

Permalink

I would like to switch to another viewController from within the ARCalView. It works fine from the OverlayViewController but I want to do it without userinteraction, e.g. when the movie is done I want to switch to another view.

junkieHead

Thu, 09/13/2012 - 08:25

Permalink

Note that ARCALView is a subclass of UIView and not UIViewCOntroller . So, if you try to present another viewcontroller modally then it will throw exception. You should add your new view as a subview of ARCALView. That should sove your issue.