Log in or register to post comments

Playing Video Over the image

August 5, 2011 - 8:35am #68

Hi All,

I understood the concept of tracking an image in Image Targets sample code.
I am able to track the image which I uploaded using Target Management Tool.
When ever image got tracked, instead of displaying another image on top of tracked image is it possible to play a video on top of the traced image using AR?

I tired to play using below code

-(void)renderFrameQCAR{
[self performSelectorOnMainThread:@selector(playVideo) withObject:nil waitUntilDone:NO];
}

-(void)playVideo{
MPMoviePlayerController *videoViewCont = [[MPMoviePlayerController alloc]initWithContentURL:[NSURL fileURLWithPath:@"WP_000019-qcif-180k.mov"]];
[videoViewCont play];
[videoViewCont.view setFrame:self.bounds];
[self addSubview:videoViewCont.view];
}

the method is getting called but video is not playing.

please give your valuable suggestions.

Re: Playing Video Over the image

August 8, 2011 - 3:34am #67

Hi shivintu,

To add additional UI, whether video views or UI controls, it's probably best practise to create a new parent view, that parents both the EAGLView and a view that contains the overlaid content.

I've done that here as a test, to play the video in a 2D pop-up overlaying the 3D/Camera view and it works fine.

Remember that the UIScreen bounds will not be in the same orientation as the sub-views in the ImageTargets case as they are forced landscape, so set the EAGLView subview frame to be [0,0,screenBounds.size.height, screenBounds.size.width].

CGRect screenBounds = [[UIScreen mainScreen] bounds];

window = [[UIWindow alloc] initWithFrame: screenBounds];
viewController = [[ARViewController alloc] init];
parentV = [[UIView alloc] initWithFrame: screenBounds];

screenBounds = CGRectMake(0, 0, parentV.frame.size.height, parentV.frame.size.width);
view = [[EAGLView alloc] initWithFrame: screenBounds];
[parentV addSubview:view];

CGRect subScreenBounds = CGRectMake(500, 100, 500, 500);
videoV = [[UIView alloc] initWithFrame:subScreenBounds];
[parentV addSubview:videoV];

[viewController setView: parentV];

Note that to play video within the 3D world you'd need to sync the video frame rate with the camera frame rate and insert each frame into the 3D view as the texture of a displayed object. This is unlikely to yield good performance as you are bypassing all the optimisation in the movie player.

Re: Playing Video Over the image

August 8, 2011 - 7:44am #66
MoSR wrote:

Hi shivintu,

To add additional UI, whether video views or UI controls, it's probably best practise to create a new parent view, that parents both the EAGLView and a view that contains the overlaid content.

I've done that here as a test, to play the video in a 2D pop-up overlaying the 3D/Camera view and it works fine.

Remember that the UIScreen bounds will not be in the same orientation as the sub-views in the ImageTargets case as they are forced landscape, so set the EAGLView subview frame to be [0,0,screenBounds.size.height, screenBounds.size.width].

CGRect screenBounds = [[UIScreen mainScreen] bounds];

window = [[UIWindow alloc] initWithFrame: screenBounds];
viewController = [[ARViewController alloc] init];
parentV = [[UIView alloc] initWithFrame: screenBounds];

screenBounds = CGRectMake(0, 0, parentV.frame.size.height, parentV.frame.size.width);
view = [[EAGLView alloc] initWithFrame: screenBounds];
[parentV addSubview:view];

CGRect subScreenBounds = CGRectMake(500, 100, 500, 500);
videoV = [[UIView alloc] initWithFrame:subScreenBounds];
[parentV addSubview:videoV];

[viewController setView: parentV];

Note that to play video within the 3D world you'd need to sync the video frame rate with the camera frame rate and insert each frame into the 3D view as the texture of a displayed object. This is unlikely to yield good performance as you are bypassing all the optimisation in the movie player.

Thank you very much MoSR, I tried it and its working, I have another problem. How we can place the video screen exactly on top of the traced image area with same width and height of traced image. and also as image is scaling (if i move the camera close to the image then phone entire screen will be occupied by image and when i am going far to the image then occupation of image on mobile real estate will be reduced) how can we scale the video screen.

I have tired below code to place the video screen on top of traced image.

const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration();
QCAR::Vec2F screenPoint = QCAR::Tool::projectPoint(cameraCalibration, trackable->getPose(), QCAR::Vec3F(0, 0, 0));
targetXPos = screenPoint.data[0];
targetYPos = screenPoint.data[1];
videoV setFrame:cgrectMake(targetXpos,targetYPos,237,150);

Here i am give width and height are static, bcz i am not able to get image width and height. i have tried QCAR::Image::getWidth(); API but as there is no static function to get the Image class instance, not getting to get the image width and height dynamically.

The video screen is not placing exactly on top of image. but its moving along with image with some rect difference as image x and y pos's are traced out.

I need help on how we can place video screen exactly on top of traced image with same width and height of traced image as well how we can scale it.

Please help me.

Re: Playing Video Over the image

August 9, 2011 - 4:05am #65

[QUOTE=shivintu" />http://ar.qualcomm.at/node/2000523>

1) Using QCAR::Image won't give you what you need as what you want is the projected size of the image in the view, not the original size of the source image. Use the 'projectionMatrix' and QCAR::Trackable::getPose() to determine the trackable target's location and orientation. See the thread:

How to export target coordinates to java?

2) This may be the latency of the video player redraw - try drawing a small coloured view instead and see how it tracks.

Re: Playing Video Over the image

August 9, 2011 - 5:47am #64

[QUOTE=MoSR" />http://ar.qualcomm.at/node/2000523How to export target coordinates to java?

2) This may be the latency of the video player redraw - try drawing a small coloured view instead and see how it tracks.[/QUOTE" />

Hi MoSR, Thank you for reply.

The code which you have provided in the attached link is I already used it, and i am able to get the x,y positions but it seems those are pointing to the center of target.

I want to place the video screen exactly on top of the traced image. i.e even i move camera close to the image or far to the image, just video player should be placed on top it.

Please help me on this, i am trying for that.

Re: Playing Video Over the image

August 9, 2011 - 6:45am #63
shivintu wrote:

I want to place the video screen exactly on top of the traced image. i.e even i move camera close to the image or far to the image, just video player should be placed on top it.

When you create the trackable, you specify a size of the trackable in your own 'internal units' (see config.xml in the project) - the scale information from the pose is relative to that.

For example if your trackable is 50x50 units then if you take a 50 by 50 quad and transform it by the projection matrix and the trackable pose, and then project those points to screen, it should give you the screen coordinates of the target.

Well that's the theory anyway - I'll try this as soon as I can and post the code as an example.

Re: Playing Video Over the image

August 9, 2011 - 8:37am #62
MoSR wrote:

When you create the trackable, you specify a size of the trackable in your own 'internal units' (see config.xml in the project) - the scale information from the pose is relative to that.

For example if your trackable is 50x50 units then if you take a 50 by 50 quad and transform it by the projection matrix and the trackable pose, and then project those points to screen, it should give you the screen coordinates of the target.

Well that's the theory anyway - I'll try this as soon as I can and post the code as an example.

Ok, That would be really helpful for me. I will also try. Thank you for spending valuable time.

Re: Playing Video Over the image

August 9, 2011 - 10:15am #61

The following will give you the four coordinates of the target corners within the EAGLView. Using the max/min in each dimension will allow you to specify another UIView that hides the target.

I used this code to display a different coloured 10x10-pixel UIView at each corner of the target. Note the adjustment for the camera view being deeper than the screen (the screen clips the camera view - this may explain why your earlier attempt was offset).

s0 thro' s3 are new public properties of type CGPoint, in EAGLView. Your delegate code can access this to reposition other views.

- (CGPoint) projectCoord:(CGPoint)coord inView:(const QCAR::CameraCalibration&)cameraCalibration andPose:(QCAR::Matrix34F)pose withOffset:(CGPoint)offset
{
    CGPoint converted;
    
    QCAR::Vec3F vec(coord.x,coord.y,0);
    QCAR::Vec2F sc = QCAR::Tool::projectPoint(cameraCalibration, pose, vec);
    converted.x = sc.data[0] - offset.x;
    converted.y = sc.data[1] - offset.y;
    
    return converted;
}

- (void) calcScreenCoordsOf:(CGSize)target inView:(CGFloat *)matrix inPose:(QCAR::Matrix34F)pose
{
    // 0,0 is at centre of target so extremities are at w/2,h/2
    CGFloat w = target.width/2;
    CGFloat h = target.height/2;
    
    const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
    const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration();

    // calculate any mismatch of screen to video size
    QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();
    QCAR::VideoMode videoMode = cameraDevice.getVideoMode(QCAR::CameraDevice::MODE_DEFAULT);
    CGPoint margin = {(videoMode.mWidth - self.frame.size.width)/2, (videoMode.mHeight - self.frame.size.height)/2};

    // now project the 4 corners of the target
    s0 = [self projectCoord:CGPointMake(-w,h) inView:cameraCalibration andPose:pose withOffset:margin];
    s1 = [self projectCoord:CGPointMake(-w,-h) inView:cameraCalibration andPose:pose withOffset:margin];
    s2 = [self projectCoord:CGPointMake(w,-h) inView:cameraCalibration andPose:pose withOffset:margin];
    s3 = [self projectCoord:CGPointMake(w,h) inView:cameraCalibration andPose:pose withOffset:margin];
}

----8<---- in renderFrameQCAR ----8<----

ShaderUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale, &modelViewMatrix.data[0]);
ShaderUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale, &modelViewMatrix.data[0]);
ShaderUtils::multiplyMatrix(&projectionMatrix.data[0], &modelViewMatrix.data[0], &modelViewProjection.data[0]);
            
 CGSize target = {247,173};
[self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

Re: Playing Video Over the image

August 9, 2011 - 10:27am #60

BTW - the target size in 3D units, that I've hard coded as {247,173}, can be recovered from the Trackable, by using it's subclass, eg. ImageTarget, see:

QCAR::ImageTarget::getSize()

Re: Playing Video Over the image

August 9, 2011 - 11:48am #59

did you have any luck with it Shivintu?

Re: Playing Video Over the image

August 9, 2011 - 10:58pm #58
MoSR wrote:

BTW - the target size in 3D units, that I've hard coded as {247,173}, can be recovered from the Trackable, by using it's subclass, eg. ImageTarget, see:

QCAR::ImageTarget::getSize()

Hi MoSR,

Thank you for posting the sample code, i tried it and even as you said i am able to get the image size using the below code and i updated CGSize target = {247,173} with dynamic sizes.

const QCAR::ImageTarget* imageTarget = static_cast(trackable);
QCAR::Vec2F::Vec2F screenPoint = imageTarget->getSize();
CGSize target;
target.width = screenPoint.data[0];
target.height = screenPoint.data[1];
// CGSize target = {247,173};
[self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

but i am unable to change the rect of "videoV" using "s0,s1,s2,s3" Could you please explain these parameters and how can we give x,y,width,height to the videoV rect.

Re: Playing Video Over the image

August 9, 2011 - 11:54pm #57
shivintu wrote:

but i am unable to change the rect of "videoV" using "s0,s1,s2,s3" Could you please explain these parameters and how can we give x,y,width,height to the videoV rect.

As I stated earlier: "Using the max/min in each dimension will allow you to specify another UIView that hides the target"

I'm in danger of writing your app for you here ;), but in pseudo-code:

x = minVal(s0.x,s1.x,s2.x,s3.x);
width = maxVal(s0.x,s1.x,s2.x,s3.x) - x;

etc. You may want to adjust the final result based on the aspect ratio of the video being played.

Re: Playing Video Over the image

August 23, 2011 - 3:59am #56

Hi shivintu,

I am also trying to play video over image but i am also getting the same problem as yours method is called but video is not playing.

Can u plz help me with this?

Re: Playing Video Over the image

August 23, 2011 - 9:50am #55
nm wrote:

I am also trying to play video over image but i am also getting the same problem as yours method is called but video is not playing.

You've probably done this, but check the the video runs in the player view in a test app before you start integrating with AR. When I was testing there were quite a few sample videos that wouldn't run on Apple's video player in iOS. I finally used the Ayrton Senna movie trailer for my tests which worked fine.

Re: Playing Video Over the image

August 25, 2011 - 2:30am #54

Thanks for reply. Yes i had tested my video and its working fine other application.

As u said in your earlier post to create a parent view which parents both EAGLEview and video view the code is as follows:

viewController = [[ARViewController alloc] init];
parentView = [[UIView alloc] initWithFrame: screenBounds];
newView = [[EAGLView alloc] initWithFrame: screenBounds];
[parentView addSubview:view];
video = [[MPMoviePlayerController alloc]initWithContentURL:[NSURL fileURLWithPath:@"teaser.mp4"]];
[video.view setFrame:CGRectMake(50, 100, 500, 500)];
[video play];
[parentView addSubview:video.view];
[viewController setView: parentV];

plz tel me where i m going wrong??

Re: Playing Video Over the image

August 25, 2011 - 3:34am #53

Hi nm,

It would be worth checking that the NSURL and 'video' are valid objects.

The path you are giving should either have its full path added or, if the video is in the bundle, you should be using something like:

NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"Senna" ofType:@"mov"]];

Also the example I gave had an intermediate view 'videoV' that I added the video view to. It may be worth adding that indirection if the above checks don't reveal anything.

Re: Playing Video Over the image

August 26, 2011 - 11:09am #52

can somebody help me with a sample example....

or tell me where I have to write the code

viewController = [[ARViewController alloc] init];
parentView = [[UIView alloc] initWithFrame: screenBounds];
newView = [[EAGLView alloc] initWithFrame: screenBounds];
[parentView addSubview:view];
NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"teaser" ofType:@"mp4"]];
video = [[MPMoviePlayerController alloc]initWithContentURL:url]];
[video.view setFrame:CGRectMake(50, 100, 500, 500)];
[video play];
[parentView addSubview:video.view];
[viewController setView: parentV];

in the ImageTargetsAppDelegate

or in EAGLView

Re: Playing Video Over the image

August 28, 2011 - 9:32am #51

That would go in the ImageTargetsAppDelegate. Something like this:

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    BOOL ret = YES;
    CGRect screenBounds = [[UIScreen mainScreen] bounds];
    
    window = [[UIWindow alloc] initWithFrame: screenBounds];
    viewController = [[ARViewController alloc] init];
    UIView* parentV = [[UIView alloc] initWithFrame: screenBounds];
    
    screenBounds = CGRectMake(0, 0, parentV.frame.size.height, parentV.frame.size.width);
    view = [[EAGLView alloc] initWithFrame: screenBounds];
    [parentV addSubview:view];
    
    CGRect subScreenBounds = CGRectMake(0, 0, 200, 200);
    NSURL* url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"myvideo" ofType:@"mp4"]];
    MPMoviePlayerController* video = [[MPMoviePlayerController alloc] initWithContentURL:url];
    [video.view setFrame:subScreenBounds];
    [parentV addSubview:video.view];
    
    [viewController setView: parentV];
    
    [window addSubview: viewController.view];
    [window makeKeyAndVisible];
    
    if (YES == ret) {
        [view onCreate];
    }
    
    return ret;
}

- Kim

Playing Video Over the image in Android

August 28, 2011 - 7:06pm #50

Hi MoSr,

it is good to know you're pushing functionality ahead !

I have already successfuly implemented the "Toast" example and called a video intent, but I would like to know how overlay the AR & video playback ( Not to include the video frames as a texture into the 3D scene ).

The problem I foresee is how to call video play within the JNI renderframe through the message handler and let it know what View to use ?

thanks, raf

Re: Playing Video Over the image

September 2, 2011 - 12:10am #49

Hi ksiva,

Thanks for replying but using your way the video starts playing when the app startsbut i want to play the video when the camera focuses on the trackable.

Plz help me with this...

Re: Playing Video Over the image

September 2, 2011 - 11:20am #48

sallespro and nm:

It seems like your questions are related (one for Android, one for iOS). In either case, I might set up the video view ahead of time, add it to the view hierarchy, hide it, and wait until a target is found to show the view and play the video. For both Android and iOS you'll need to make sure that you change the view visibility on the main thread. The toast sample (http://ar.qualcomm.at/node/2000032) shows how to do this on Android using a Handler. On iOS, you can use the performSelectorOnMainThread method.

So for the iOS code above, you can add this to the didFinishLaunchingWithOptions method:

view.movieController = video;
video.view.hidden = YES;
video.shouldAutoplay = NO;

(Set up a property in the EAGLView to store the MPMoviePlayerController object.)

Then, in EAGLView.mm you can add something like this:

bool videoPlaying = NO;

- (void)showMovieController
{
    movieController.view.hidden = NO;
}

- (void)renderFrameQCAR
{
    ...
    for (int i = 0; i < state.getNumActiveTrackables(); ++i)
    {
        ...
        if (!videoPlaying)
        {
            [self performSelectorOnMainThread:@selector(showMovieController) withObject:nil waitUntilDone:NO];
            videoPlaying = YES;
        }

        ...
    }
    ...
}

- Kim

Re: Playing Video Over the image

January 16, 2012 - 4:56am #47

can you specify how i can store the moviecontroller in the eaglview?

i have a method called - (void)showMovieController in my appdelegate where i switch on the video but when i call it via selector in my eaglview it crashes cause it does not find this method.

Re: Playing Video Over the image

January 16, 2012 - 5:57am #46

Hi iossif,

Note that the 'performSelectorOnMainThread' doesn't have a 'target' parameter so the method called has to be in the same class.

To call a selector in another class you need to get the class instance and call the method on that. Here's how when that class is the app delegate:

YourDelegate *appDelegate = (YourDelegate *)[[UIApplication sharedApplication] delegate];
[appDelegate performSelectorOnMainThread:... ];

Re: Playing Video Over the image

February 6, 2012 - 10:19pm #45

hi, question about syncing the video frame rate with the camera frame rate.

MoSR wrote:

Note that to play video within the 3D world you'd need to sync the video frame rate with the camera frame rate and insert each frame into the 3D view as the texture of a displayed object. This is unlikely to yield good performance as you are bypassing all the optimisation in the movie player.

how exactly can this be done?

thanks,
L.

Re: Playing Video Over the image

February 7, 2012 - 3:30am #44

Take a look at iOS AVURLAsset - there are examples from Apple and 3rd parties. Getting the frames is well documented - sync'ing with renderFrameQCAR isn't and we've not yet tried it (but will in the future). One way would be to store frames in a pipe (FIFO) so that in renderFrameQCAR you can pull the frames out and show the one that sync's with the actual time.

Re: Playing Video Over the image

February 27, 2012 - 12:16pm #43

Hi,

I tried to implement the code as shown below into AR_EAGLView.mm class, but the QCAR::Tracker, QCAR::CameraDevice and QCAR::VideoMode classes are not found. Error from XCode: No member named "CameraDevice" in Namespace QCAR. I guess i have to use a simple import command for the classes, but i dont know which one..

EDIT:

by including

#import
#import
#import

some errors dissapeared. But QCAR::Tracker::getInstance() is undefined, as well as tracker.getCameraCalibration();

EDIT 2 the example from MoSR seems to be outdated, as the API shows no tracker.getInstance() anymore. my updated approach:

CGFloat w = target.width/2;
CGFloat h = target.height/2;

// calculate any mismatch of screen to video size
QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();

const QCAR::CameraCalibration& cameraCalibration = cameraDevice.getCameraCalibration();

MoSR wrote:

The following will give you the four coordinates of the target corners within the EAGLView. Using the max/min in each dimension will allow you to specify another UIView that hides the target.

I used this code to display a different coloured 10x10-pixel UIView at each corner of the target. Note the adjustment for the camera view being deeper than the screen (the screen clips the camera view - this may explain why your earlier attempt was offset).

s0 thro' s3 are new public properties of type CGPoint, in EAGLView. Your delegate code can access this to reposition other views.

- (CGPoint) projectCoord:(CGPoint)coord inView:(const QCAR::CameraCalibration&)cameraCalibration andPose:(QCAR::Matrix34F)pose withOffset:(CGPoint)offset
{
    CGPoint converted;
    
    QCAR::Vec3F vec(coord.x,coord.y,0);
    QCAR::Vec2F sc = QCAR::Tool::projectPoint(cameraCalibration, pose, vec);
    converted.x = sc.data[0] - offset.x;
    converted.y = sc.data[1] - offset.y;
    
    return converted;
}

- (void) calcScreenCoordsOf:(CGSize)target inView:(CGFloat *)matrix inPose:(QCAR::Matrix34F)pose
{
    // 0,0 is at centre of target so extremities are at w/2,h/2
    CGFloat w = target.width/2;
    CGFloat h = target.height/2;
    
    const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
    const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration();

    // calculate any mismatch of screen to video size
    QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();
    QCAR::VideoMode videoMode = cameraDevice.getVideoMode(QCAR::CameraDevice::MODE_DEFAULT);
    CGPoint margin = {(videoMode.mWidth - self.frame.size.width)/2, (videoMode.mHeight - self.frame.size.height)/2};

    // now project the 4 corners of the target
    s0 = [self projectCoord:CGPointMake(-w,h) inView:cameraCalibration andPose:pose withOffset:margin];
    s1 = [self projectCoord:CGPointMake(-w,-h) inView:cameraCalibration andPose:pose withOffset:margin];
    s2 = [self projectCoord:CGPointMake(w,-h) inView:cameraCalibration andPose:pose withOffset:margin];
    s3 = [self projectCoord:CGPointMake(w,h) inView:cameraCalibration andPose:pose withOffset:margin];
}

----8<---- in renderFrameQCAR ----8<----

ShaderUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale, &modelViewMatrix.data[0]);
ShaderUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale, &modelViewMatrix.data[0]);
ShaderUtils::multiplyMatrix(&projectionMatrix.data[0], &modelViewMatrix.data[0], &modelViewProjection.data[0]);
            
 CGSize target = {247,173};
[self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

Re: Playing Video Over the image

March 1, 2012 - 4:30am #42

Folowing sommeralex's observations I've updated the sample code for tracking the corners of a target, to be compatible with QCAR iOS SDK 1.0 release (rather than beta), shown below. In the beta there was an additional view controller which was optimised out in the release. I'll create a new sample for QCAR iOS SDK 1.5.

For QCAR iOS SDK 1.0:

We're going to modify the ImageTargets sample app to draw a coloured flag on each corner of the detected target.

ImageTargetsAppDelegate.h add:

    UIView *s0V;
    UIView *s1V;
    UIView *s2V;
    UIView *s3V;

ImageTargetsAppDelegate.mm add:

    // Add the EAGLView and the overlay view to the window
    [window addSubview:view];
    

    // add a mini view for each target corner in a parent view
    UIView *parentV = [[UIView alloc] initWithFrame:viewBounds];
    CGRect frame = {0,0,10,10};
    s0V = [[UIView alloc] initWithFrame:frame];
    s1V = [[UIView alloc] initWithFrame:frame];
    s2V = [[UIView alloc] initWithFrame:frame];
    s3V = [[UIView alloc] initWithFrame:frame];
    s0V.backgroundColor = [UIColor redColor];
    s1V.backgroundColor = [UIColor greenColor];
    s2V.backgroundColor = [UIColor blueColor];
    s3V.backgroundColor = [UIColor yellowColor];
    [parentV addSubview:s0V];
    [parentV addSubview:s1V];
    [parentV addSubview:s2V];
    [parentV addSubview:s3V];
    
    // rotate the parent view to match the camera view orientation
    parentV.layer.position = pos;
    parentV.transform = rotate;
    
    [window addSubview:parentV];
    
    [window addSubview: overlayViewController.view];
    [window makeKeyAndVisible];
    
    // Perform actions on the EAGLView now it has been created
    [view onCreate];

    // for this demo update the target corners on a simple timer 
    [NSTimer scheduledTimerWithTimeInterval:0.04 target:self selector:@selector(moveCorners:) userInfo:NULL repeats:YES];

Plus the timer callback function:

- (void)moveCorners:(NSTimer*)theTimer
{
    CGRect frame = CGRectMake(view.s0.x, view.s0.y, 10, 10);
    s0V.frame = frame;
    frame = CGRectMake(view.s1.x, view.s1.y, 10, 10);
    s1V.frame = frame;
    frame = CGRectMake(view.s2.x, view.s2.y, 10, 10);
    s2V.frame = frame;
    frame = CGRectMake(view.s3.x, view.s3.y, 10, 10);
    s3V.frame = frame;
}

In EAGLView.h add the following properties to the interface in the usual places:

@public
    CGPoint s0;
    CGPoint s1;
    CGPoint s2;
    CGPoint s3;

@property (nonatomic)     CGPoint s0;
@property (nonatomic)     CGPoint s1;
@property (nonatomic)     CGPoint s2;
@property (nonatomic)     CGPoint s3;

And in EAGLView.mm synthesize the properties, import QCAR/Vectors.h, and add the following (noting the change in which dimension is now used for calculating the offset):

- (CGPoint) projectCoord:(CGPoint)coord inView:(const QCAR::CameraCalibration&)cameraCalibration andPose:(QCAR::Matrix34F)pose withOffset:(CGPoint)offset andScale:(CGFloat)scale
{
    CGPoint converted;
    
    QCAR::Vec3F vec(coord.x,coord.y,0);
    QCAR::Vec2F sc = QCAR::Tool::projectPoint(cameraCalibration, pose, vec);
    converted.x = sc.data[0]*scale - offset.x;
    converted.y = sc.data[1]*scale - offset.y;
    
    return converted;
}

- (void) calcScreenCoordsOf:(CGSize)target inView:(CGFloat *)matrix inPose:(QCAR::Matrix34F)pose
{
    // 0,0 is at centre of target so extremities are at w/2,h/2
    CGFloat w = target.width/2;
    CGFloat h = target.height/2;
    
    // need to account for the orientation on view size
    CGFloat viewWidth = self.frame.size.height; // Portrait
    CGFloat viewHeight = self.frame.size.width; // Portrait    
    UIInterfaceOrientation orientation = [UIApplication sharedApplication].statusBarOrientation;
    if (UIInterfaceOrientationIsLandscape(orientation))
    {
        viewWidth = self.frame.size.width;
        viewHeight = self.frame.size.height;        
    }
    
    // calculate any mismatch of screen to video size
    QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();
    const QCAR::CameraCalibration& cameraCalibration = cameraDevice.getCameraCalibration();
    QCAR::VideoMode videoMode = cameraDevice.getVideoMode(QCAR::CameraDevice::MODE_DEFAULT);
    
    CGFloat scale = viewWidth/videoMode.mWidth;
    if (videoMode.mHeight * scale < viewHeight)
        scale = viewHeight/videoMode.mHeight;
    CGFloat scaledWidth = videoMode.mWidth * scale;
    CGFloat scaledHeight = videoMode.mHeight * scale;
        
    CGPoint margin = {(scaledWidth - viewWidth)/2, (scaledHeight - viewHeight)/2};
    
    // now project the 4 corners of the target
    ImageTargetsAppDelegate *delegate = [[UIApplication sharedApplication] delegate];
    delegate.s0 = [self projectCoord:CGPointMake(-w,h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
    delegate.s1 = [self projectCoord:CGPointMake(-w,-h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
    delegate.s2 = [self projectCoord:CGPointMake(w,-h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
    delegate.s3 = [self projectCoord:CGPointMake(w,h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
}

And in renderFrameQCAR, during the tracking loop add:

            ShaderUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale, &modelViewMatrix.data[0]);
            ShaderUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale, &modelViewMatrix.data[0]);
            ShaderUtils::multiplyMatrix(&projectionMatrix.data[0], &modelViewMatrix.data[0], &modelViewProjection.data[0]);

            CGSize target = {247,173};
            [self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

Re: Playing Video Over the image

March 1, 2012 - 7:53am #41

For QCAR iOS SDK 1.5 (without the overlayed object autorotating - I'll spin another version later where the corner tracking view is always in the user's orientation, and the coordinates match):

In this sample I'll keep the coordinates in the delegate as it's somewhat easier to get to that than it is for the delegate to navigate up the more complex view hierarchy.

In ImageTargetsAppDelegate.h:

@interface ImageTargetsAppDelegate : NSObject <UIApplicationDelegate> {
    UIWindow* window;
    ARParentViewController* arParentViewController;
    UIImageView *splashV;

    UIView *s0V;
    UIView *s1V;
    UIView *s2V;
    UIView *s3V;
    
@public
    CGPoint s0;
    CGPoint s1;
    CGPoint s2;
    CGPoint s3;

}

@property (nonatomic)     CGPoint s0;
@property (nonatomic)     CGPoint s1;
@property (nonatomic)     CGPoint s2;
@property (nonatomic)     CGPoint s3;

In ImageTargetsAppDelegate.mm synthesize the properties and add the following imports:

#import <QCAR/CameraDevice.h>

In application:didFinishLaunching... insert the following to set up the views:

// Add the EAGLView and the overlay view to the window
arParentViewController = [[ARParentViewController alloc] init];
arParentViewController.arViewRect = screenBounds;
[window insertSubview:arParentViewController.view atIndex:0];

CGRect frame = {0,0,10,10};
s0V = [[UIView alloc] initWithFrame:frame];
s1V = [[UIView alloc] initWithFrame:frame];
s2V = [[UIView alloc] initWithFrame:frame];
s3V = [[UIView alloc] initWithFrame:frame];
s0V.backgroundColor = [UIColor redColor];
s1V.backgroundColor = [UIColor greenColor];
s2V.backgroundColor = [UIColor blueColor];
s3V.backgroundColor = [UIColor yellowColor];

frame = CGRectMake(0, 0, screenBounds.size.height, screenBounds.size.width);
UIView *cornerTrackingV = [[UIView alloc] initWithFrame:frame];
[cornerTrackingV addSubview:s0V];
[cornerTrackingV addSubview:s1V];
[cornerTrackingV addSubview:s2V];
[cornerTrackingV addSubview:s3V];

CGPoint pos;
pos.x = screenBounds.size.width / 2;
pos.y = screenBounds.size.height / 2;
CGAffineTransform rotate = CGAffineTransformMakeRotation(90 * M_PI  / 180);
cornerTrackingV.layer.position = pos;
cornerTrackingV.transform = rotate;

[window addSubview:cornerTrackingV];

[window makeKeyAndVisible];

And add the timer callback method to move the views to the new coordinates:

- (void)moveCorners:(NSTimer*)theTimer
{
    CGRect frame = CGRectMake(s0.x, s0.y, 10, 10);
    s0V.frame = frame;
    frame = CGRectMake(s1.x, s1.y, 10, 10);
    s1V.frame = frame;
    frame = CGRectMake(s2.x, s2.y, 10, 10);
    s2V.frame = frame;
    frame = CGRectMake(s3.x, s3.y, 10, 10);
    s3V.frame = frame;
}

In EAGLView.mm, add the two methods for calculating the target corner projections:

- (CGPoint) projectCoord:(CGPoint)coord inView:(const QCAR::CameraCalibration&)cameraCalibration andPose:(QCAR::Matrix34F)pose withOffset:(CGPoint)offset andScale:(CGFloat)scale
{
    CGPoint converted;
    
    QCAR::Vec3F vec(coord.x,coord.y,0);
    QCAR::Vec2F sc = QCAR::Tool::projectPoint(cameraCalibration, pose, vec);
    converted.x = sc.data[0]*scale - offset.x;
    converted.y = sc.data[1]*scale - offset.y;
    
    return converted;
}

- (void) calcScreenCoordsOf:(CGSize)target inView:(CGFloat *)matrix inPose:(QCAR::Matrix34F)pose
{
    // 0,0 is at centre of target so extremities are at w/2,h/2
    CGFloat w = target.width/2;
    CGFloat h = target.height/2;
    
    // need to account for the orientation on view size
    CGFloat viewWidth = self.frame.size.height; // Portrait
    CGFloat viewHeight = self.frame.size.width; // Portrait    
    UIInterfaceOrientation orientation = [UIApplication sharedApplication].statusBarOrientation;
    if (UIInterfaceOrientationIsLandscape(orientation))
    {
        viewWidth = self.frame.size.width;
        viewHeight = self.frame.size.height;        
    }
    
    // calculate any mismatch of screen to video size
    QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();
    const QCAR::CameraCalibration& cameraCalibration = cameraDevice.getCameraCalibration();
    QCAR::VideoMode videoMode = cameraDevice.getVideoMode(QCAR::CameraDevice::MODE_DEFAULT);
    
    CGFloat scale = viewWidth/videoMode.mWidth;
    if (videoMode.mHeight * scale < viewHeight)
        scale = viewHeight/videoMode.mHeight;
    CGFloat scaledWidth = videoMode.mWidth * scale;
    CGFloat scaledHeight = videoMode.mHeight * scale;
        
    CGPoint margin = {(scaledWidth - viewWidth)/2, (scaledHeight - viewHeight)/2};
    
    // now project the 4 corners of the target
    ImageTargetsAppDelegate *delegate = [[UIApplication sharedApplication] delegate];
    delegate.s0 = [self projectCoord:CGPointMake(-w,h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
    delegate.s1 = [self projectCoord:CGPointMake(-w,-h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
    delegate.s2 = [self projectCoord:CGPointMake(w,-h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
    delegate.s3 = [self projectCoord:CGPointMake(w,h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];
}

These are invoked from within renderFrameQCAR:

            ShaderUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale, &modelViewMatrix.data[0]);
            ShaderUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale, &modelViewMatrix.data[0]);
            ShaderUtils::multiplyMatrix(&qUtils.projectionMatrix.data[0], &modelViewMatrix.data[0], &modelViewProjection.data[0]);
            
            CGSize target = {247,173};
            [self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];
            
            glUseProgram(shaderProgramID);

Re: Playing Video Over the image

March 2, 2012 - 10:37am #40

On QCAR iOS 1.5 with the UIView attached to the target and autorotated:

We're going to demonstrate drawing a view at each of the corner points of a target, and a scaled view central to the target, all of which will stay oriented correctly.

Create a subclass of ARParentViewController (CornerTrackingVC) to insert a new layer between the ARView and the popup Overlay View:

#import <UIKit/UIKit.h>
#import "ARParentViewController.h"

@interface CornerTrackingVC : ARParentViewController
{
    UIView *s0V;
    UIView *s1V;
    UIView *s2V;
    UIView *s3V;
    UIView *centreV;
}

@property (nonatomic, assign) UIView *s0V;
@property (nonatomic, assign) UIView *s1V;
@property (nonatomic, assign) UIView *s2V;
@property (nonatomic, assign) UIView *s3V;
@property (nonatomic, assign) UIView *centreV;

@end

And:

#import <QuartzCore/QuartzCore.h>
#import "CornerTrackingVC.h"
#import "ARViewController.h"
#import "OverlayViewController.h"

@implementation CornerTrackingVC

@synthesize s0V;
@synthesize s1V;
@synthesize s2V;
@synthesize s3V;
@synthesize centreV;

- (void) loadView
{
    NSLog(@"ARParentVC: creating");
    parentView = [[UIView alloc] initWithFrame:arViewRect];
    parentView.autoresizesSubviews = YES;
    
    // Add the EAGLView and the overlay view to the window
    arViewController = [[ARViewController alloc] init];
    
    // need to set size here to setup camera image size for AR
    arViewController.arViewSize = arViewRect.size;
    [parentView addSubview:arViewController.view];
    
    // These are the four corner points
    CGRect frame = {0,0,4,4};
    s0V = [[UIView alloc] initWithFrame:frame];
    s1V = [[UIView alloc] initWithFrame:frame];
    s2V = [[UIView alloc] initWithFrame:frame];
    s3V = [[UIView alloc] initWithFrame:frame];
    
    s0V.backgroundColor = [UIColor redColor];
    s1V.backgroundColor = [UIColor greenColor];
    s2V.backgroundColor = [UIColor blueColor];
    s3V.backgroundColor = [UIColor yellowColor];

    // And this is the centre-piece
    centreV = [[UIView alloc] initWithFrame:CGRectMake(20, 50, 4, 4)]; 
    centreV.backgroundColor = [UIColor orangeColor];
    [parentView addSubview:centreV];
    
    // Create a container that is correctly resized and rotated
    frame = CGRectMake(0, 0, arViewRect.size.width, arViewRect.size.height);
    UIView *cornerTrackingV = [[UIView alloc] initWithFrame:frame];
    [cornerTrackingV addSubview:s0V];
    [cornerTrackingV addSubview:s1V];
    [cornerTrackingV addSubview:s2V];
    [cornerTrackingV addSubview:s3V];
    cornerTrackingV.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
    [parentView addSubview:cornerTrackingV];
     
    // Create an auto-rotating overlay view and its view controller (used for
    // displaying UI objects, such as the camera control menu)
    overlayViewController = [[OverlayViewController alloc] init];
    [parentView addSubview: overlayViewController.view];
    
    self.view = parentView;
}

- (void)dealloc
{
    self.s0V = nil;
    self.s1V = nil;
    self.s2V = nil;
    self.s3V = nil;
    self.centreV = nil;
    
    [super dealloc];
}

In the app delegate, hold the coordinates and ref the subclass:

#import <UIKit/UIKit.h>
@class CornerTrackingVC;

@interface ImageTargetsAppDelegate : NSObject <UIApplicationDelegate> {
    UIWindow* window;
    CornerTrackingVC* cornerTrackingVC;
    UIImageView *splashV;

@public
    CGPoint s0;
    CGPoint s1;
    CGPoint s2;
    CGPoint s3;
    CGPoint centre;

}

@property (nonatomic)     CGPoint s0;
@property (nonatomic)     CGPoint s1;
@property (nonatomic)     CGPoint s2;
@property (nonatomic)     CGPoint s3;
@property (nonatomic)     CGPoint centre;

@end

Set up the subclass in place of ARParentViewController, and add a timer for the animation:

    // Add the EAGLView and the overlay view to the window
    cornerTrackingVC = [[CornerTrackingVC alloc] init];
    cornerTrackingVC.arViewRect = screenBounds;
    [window insertSubview:cornerTrackingVC.view atIndex:0];

    [window makeKeyAndVisible];

    [NSTimer scheduledTimerWithTimeInterval:0.04 target:self selector:@selector(moveCorners:) userInfo:NULL repeats:YES];

And add the method to manipulate the views per time period, using the target coordinates:

- (void)moveCorners:(NSTimer*)theTimer
{
    // move each of the corner 'tags'
    CGRect frame = CGRectMake(s0.x, s0.y, 4, 4);
    cornerTrackingVC.s0V.frame = frame;
    frame = CGRectMake(s1.x, s1.y, 4, 4);
    cornerTrackingVC.s1V.frame = frame;
    frame = CGRectMake(s2.x, s2.y, 4, 4);
    cornerTrackingVC.s2V.frame = frame;
    frame = CGRectMake(s3.x, s3.y, 4, 4);
    cornerTrackingVC.s3V.frame = frame;
    
    // simplistically make the scale the average of the two diagonals
    // we could extract the scale from the pose
    CGFloat diag1 = sqrtf(powf(s0.x-s2.x,2.0) + powf(s0.y-s2.y,2.0));
    CGFloat diag2 = sqrtf(powf(s1.x-s3.x,2.0) + powf(s1.y-s3.y,2.0));

    // draw a 4:3 rectangle scaled to the target 
    frame.size.width = (diag1 + diag2)/2 / 2.0;
    frame.size.height = frame.size.width * 3.0/4.0;
    frame.origin.x = centre.x - frame.size.width/2;
    frame.origin.y = centre.y - frame.size.height/2;
    
    cornerTrackingVC.centreV.frame = frame;
}

In the EAGLView add methods that calculate the correct positions of target points, accounting for the screen orientation:

- (CGPoint) projectCoord:(CGPoint)coord inView:(const QCAR::CameraCalibration&)cameraCalibration andPose:(QCAR::Matrix34F)pose withOffset:(CGPoint)offset andScale:(CGFloat)scale
{
    CGPoint converted;
    
    QCAR::Vec3F vec(coord.x,coord.y,0);
    QCAR::Vec2F sc = QCAR::Tool::projectPoint(cameraCalibration, pose, vec);
    converted.x = sc.data[0]*scale - offset.x;
    converted.y = sc.data[1]*scale - offset.y;
    
    return converted;
}

- (CGPoint) toPortrait:(CGPoint) coord
{
    CGPoint newCoord;
    newCoord.x = self.frame.size.width - coord.y;
    newCoord.y = coord.x;
    
    return newCoord;
}

- (CGPoint) toPortraitUD:(CGPoint) coord
{
    CGPoint newCoord;
    newCoord.x = coord.y;
    newCoord.y = self.frame.size.height - coord.x;
    
    return newCoord;
}

- (CGPoint) toLandscapeLeft:(CGPoint) coord
{
    CGPoint newCoord;
    newCoord.x = self.frame.size.width - coord.x;
    newCoord.y = self.frame.size.height - coord.y;
    
    return newCoord;
}

- (void) calcScreenCoordsOf:(CGSize)target inView:(CGFloat *)matrix inPose:(QCAR::Matrix34F)pose
{
    // 0,0 is at centre of target so extremities are at w/2,h/2
    CGFloat w = target.width/2;
    CGFloat h = target.height/2;
    
    // need to account for the orientation on view size
    CGFloat viewWidth = self.frame.size.height; // Portrait
    CGFloat viewHeight = self.frame.size.width; // Portrait    
    UIInterfaceOrientation orientation = [UIApplication sharedApplication].statusBarOrientation;
    if (UIInterfaceOrientationIsLandscape(orientation))
    {
        viewWidth = self.frame.size.width;
        viewHeight = self.frame.size.height;        
    }
    
    // calculate any mismatch of screen to video size
    QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();
    const QCAR::CameraCalibration& cameraCalibration = cameraDevice.getCameraCalibration();
    QCAR::VideoMode videoMode = cameraDevice.getVideoMode(QCAR::CameraDevice::MODE_DEFAULT);
    
    CGFloat scale = viewWidth/videoMode.mWidth;
    if (videoMode.mHeight * scale < viewHeight)
        scale = viewHeight/videoMode.mHeight;
    CGFloat scaledWidth = videoMode.mWidth * scale;
    CGFloat scaledHeight = videoMode.mHeight * scale;
        
    CGPoint margin = {(scaledWidth - viewWidth)/2, (scaledHeight - viewHeight)/2};
    
    // now project the 4 corners of the target
    ImageTargetsAppDelegate *delegate = [[UIApplication sharedApplication] delegate];
    delegate.s0 = [self projectCoord:CGPointMake(-w,h) inView:cameraCalibration andPose:pose withOffset:margin andScale:scale];

... ditto for s1-s3 and centre using (-w,-h), (w,-h), (w,h) and (0,0) ...
    
    // correct the coordinates for screen orientation
    switch (orientation) {
        case UIInterfaceOrientationPortrait:
            delegate.s0 = [self toPortrait:delegate.s0];
            delegate.s1 = [self toPortrait:delegate.s1];
            delegate.s2 = [self toPortrait:delegate.s2];
            delegate.s3 = [self toPortrait:delegate.s3];
            delegate.centre = [self toPortrait:delegate.centre];
            break;
            
        case UIInterfaceOrientationPortraitUpsideDown:
            delegate.s0 = [self toPortraitUD:delegate.s0];
            delegate.s1 = [self toPortraitUD:delegate.s1];
            delegate.s2 = [self toPortraitUD:delegate.s2];
            delegate.s3 = [self toPortraitUD:delegate.s3];
            delegate.centre = [self toPortraitUD:delegate.centre];
            break;
            
        case UIInterfaceOrientationLandscapeLeft:
            delegate.s0 = [self toLandscapeLeft:delegate.s0];
            delegate.s1 = [self toLandscapeLeft:delegate.s1];
            delegate.s2 = [self toLandscapeLeft:delegate.s2];
            delegate.s3 = [self toLandscapeLeft:delegate.s3];
            delegate.centre = [self toLandscapeLeft:delegate.centre];
            break;
            
        default:
            break;
    }
}

And in renderFrameQCAR add the code to the loop over detected trackables:

            CGSize target = {247,173};
            [self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

Re: Playing Video Over the image

March 5, 2012 - 2:06pm #39

Hello, when i paste code to my didFinishLaunchingWithOptions

i get error. You can see which error i get

http://www.haai.tv/error.png
Thanx for help

Re: Playing Video Over the image

March 7, 2012 - 2:56am #38

Hi Sentio - maybe you didn't start with the ImageTargets sample app? (And maybe I didn't explicitly say you should, sorry).

Re: Playing Video Over the image

March 8, 2012 - 3:15am #37

Hi MoSR,

I copied your code from posting "For QCAR iOS SDK 1.5 (without the overlayed object autorotating..." and everything works, but:
the overlayed objects rotate, just like in the follow up example (On QCAR iOS 1.5 with the UIView attached to the target and autorotated).

Any ideas?

Regards,
choffmann

Re: Playing Video Over the image

March 12, 2012 - 7:43am #36

Hi choffman,

If in both examples you set the block size to 10x30 in moveCorners you'll see the situation more clearly.

One example keeps the augmented corner shapes oriented with the camera view (not the world), the other auto-rotates those shapes with the UI. So if you are showing text or a movie it will keep in an orientation where the user can still appreciate the content.

Re: Playing Video Over the image

March 23, 2012 - 12:26pm #35

Ah, I understand that now.

But now that the position is correct, how to apply the correct rotation and the correct scale of the pose to the uiview objects?

Thanks and regards,
choffmann

Re: Playing Video Over the image

March 26, 2012 - 3:13am #34

Hi choffman,

It depends what effect you want to achieve.

1) UIViews stay fixed to "landscape left"

2) UIViews stay rectilinear with phone orientation, or

3) UIViews rotate arbitrarily with the target

The second example I've provided does 2) and for 3) you need to would use the first example but alter the way the corner information is used to position the UIView. If you calculate an angle from the coordinates and apply that to the CATransform of the UIView's CGLayer the view should rotate with the target. I've created a simple scaling calculation in the second sample.

Re: Playing Video Over the image

March 26, 2012 - 11:10am #33

http://dl.dropbox.com/u/55338579/Screenshots/IMG_0101.PNG
http://dl.dropbox.com/u/55338579/Screenshots/IMG_0102.PNG

Is there no chance to apply the GL 4x4 pose to the layer? Basically that should be possible (CATransform3D is also a 4x4 matrix), but this gives also strange results!

Any ideas?

Regards,
choffmann

Re: Playing Video Over the image

March 27, 2012 - 7:54am #32

MoSR:

I'm trying to overlay an image onto the target image that sticks to the corners od the target and stretches/shrinks depending on the angle of the camera.

I used this code to get an orange center view, but this view doesn't stick to the corners of the tracking point. It does, but when I turn the camera to the tracker from the side, the orange view is perpendicular to the target, not sticking to it.

How do I make the image stick to the target?
I'm using Qualcomm V1.5

Thanks in advance!

Re: Playing Video Over the image

March 27, 2012 - 8:02am #31

Sorry, realized I needed to implement
For QCAR iOS SDK 1.5 (without the overlayed object autorotating)

... Will keep posted on progress

Re: Playing Video Over the image

March 27, 2012 - 8:08am #30

@choffman:

First of all does it work okay when you are just scaling, not rotating? Does the view take on the size you want? (Rotation won't alter the size of the view).

If so you then need to think about the rotation centre-point, where 0 degrees is and what constitutes a positive angle. You are using the angle of the s0-s3 line - does that orient the view correctly, or do you need to be adding 90, 180 or 270 degrees?

Re: Playing Video Over the image

March 27, 2012 - 8:20am #29

@Supereid86:

There are many possible behaviour models here, amongst which are:

1) Static to the phone screen, rectilinear to the screen, and 2D
2) Auto-rotated with the UI, rectilinear to the screen and 2D
3) Arbitrarily rotated with the target, but still 2D
4) Arbitrarily rotated with the target in 3D (similar to a planar OpenGL augmentation)

1) and 2) are given in the earlier posted examples

3) is where choffman is trying to get to, above, and is just a matter of applying the right geometry.

4) requires a mapping of the pose-Matrix and projection matrix into a CATransform.

To be honest I've not got very far with 4) but not spent a lot of time on it either. I can get the target exhibiting the same movement as the target, but not superimposed precisely over the target. I think I need to understand CATransform a little better.

Anybody who knows how to do this, please pipe up! :-)

Re: Playing Video Over the image

March 27, 2012 - 9:06am #28

@MoSR

Thanks for the help,

I think I am in a good place to ask a question.

I want to create a 2D plane on the tracker area, and overlay a 2D image on top, that sticks to the tracker image 100%, regardless of camera position, and always covers it's entire area. I read in some other threads that this can be done by creating a 2D plane and overlaying a texture on top. ?
I want to do everything in 2D.

It sounds like what I need to do is your suggestion 3:
3) Arbitrarily rotated with the target, but still 2D.

As you can see in the image below, the corners of the image I overlay does not match the corners of the tracker image.

What I also don't understand is why the other views (the small squares in the corners) don't line up perfectly with the tracker image.

Thanks in advance!

Re: Playing Video Over the image

March 27, 2012 - 9:21am #27

Hi again,

(I think there's a little confusion over the use of 2D here - I effectively mean "on the plane of the screen" whereas I think you mean "on a 2D plane within the 3D world")

If you only want an image to appear fixed to the target, and not an interactive UIView, then you should implement that in OpenGL within renderFrameQCAR. Make your augmentation a 2D plane (that is, replace the teapot with a rectangle) and use your image as its texture.

Re: Playing Video Over the image

March 27, 2012 - 10:08am #26
MoSR wrote:

Hi again,

(I think there's a little confusion over the use of 2D here - I effectively mean "on the plane of the screen" whereas I think you mean "on a 2D plane within the 3D world")

If you only want an image to appear fixed to the target, and not an interactive UIView, then you should implement that in OpenGL within renderFrameQCAR. Make your augmentation a 2D plane (that is, replace the teapot with a rectangle) and use your image as its texture.

That is exactly what I want to do. I am a complete openGL beginner and don't know how to make my augmentation a 2D plane or add a texture.
Can you point me in the right direction?

Thanks so much MoSR.

Re: Playing Video Over the image

March 27, 2012 - 10:19am #25

In that case I need to point you at Phil Rideout's book on "iPhone 3D Programming" by O'Reilly, or at Jeff Lamarche's blog at iPhoneDevelopment.blogspot.com. There are other good OpenGL Tutorials out there, but we can't do this by posts on a forum, sorry. :-)

Re: Playing Video Over the image

April 9, 2012 - 12:49pm #24

I tried the following
-(void)renderFrameQCAR{
[self performSelectorOnMainThread:@selector(playVideo) withObject:nil waitUntilDone:NO];
}

-(void)playVideo{
MPMoviePlayerController *videoViewCont = [[MPMoviePlayerController alloc]initWithContentURL:[NSURL fileURLWithPath:@"testvideo.flv"]];
[videoViewCont play];
[videoViewCont.view setFrame:self.bounds];
[self addSubview:videoViewCont.view];
}

but it give me this error "undefined symbols for architecture armv7" for MPMoviePlayerController

Re: Playing Video Over the image

April 10, 2012 - 9:28am #23

@Supereid86

On your question about the squares not lining up with the target corners is probably because you need to change the following to match your target size,

CGSize target = {247,173};

or read the dimensions direct from the target object using QCAR::ImageTarget::getSize()

Re: Playing Video Over the image

April 13, 2012 - 5:06am #22

FYI - I'm about to edit my earlier posted examples to fix a scaling problem on the iPad.

Re: Playing Video Over the image

April 22, 2012 - 2:09pm #21

Hello,

after some work i can change video, image with teapot.

My problem is how can i remove points when it' is not tracking.

when i turn to camera somewhere else from target. points and shape stay on screen.

I found place where disable vertex and etc.

How can i call from this sub to removesubview for CornerTracking.m

thank for help

Re: Playing Video Over the image

April 24, 2012 - 5:28am #20

Hi Sentio,

You need some logic to handle the undetect case. You may want to timeout the lack of detection, rather than for it dissappear straight away. Reset a timer every detection, and if it times out then you've lost detection. The length of the timer will depend on the required behaviour, and whether you want to bridge short periods of non-detection.

You may also want to use the pose to animate your object, so that when you stop detecting the augmentation glides to a stop; or if it redetects it animates to the new position rather than jumping.

Once you've decided that the target is not showing you can message your overlay view code to hide, remove itself from the view hierarchy and/or destruct.

Re: Playing Video Over the image

May 8, 2012 - 1:54am #19

Hi all,
can someone upload an example code? I have tried to implement but there are always some problems. :confused:

Re: Playing Video Over the image

May 9, 2012 - 2:18am #18

where to add this code

Log in or register to post comments