Log in or register to post comments

Re: Playing Video Over the image

August 28, 2011 - 9:32am #18

That would go in the ImageTargetsAppDelegate. Something like this:

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    BOOL ret = YES;
    CGRect screenBounds = [[UIScreen mainScreen] bounds];
    
    window = [[UIWindow alloc] initWithFrame: screenBounds];
    viewController = [[ARViewController alloc] init];
    UIView* parentV = [[UIView alloc] initWithFrame: screenBounds];
    
    screenBounds = CGRectMake(0, 0, parentV.frame.size.height, parentV.frame.size.width);
    view = [[EAGLView alloc] initWithFrame: screenBounds];
    [parentV addSubview:view];
    
    CGRect subScreenBounds = CGRectMake(0, 0, 200, 200);
    NSURL* url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"myvideo" ofType:@"mp4"]];
    MPMoviePlayerController* video = [[MPMoviePlayerController alloc] initWithContentURL:url];
    [video.view setFrame:subScreenBounds];
    [parentV addSubview:video.view];
    
    [viewController setView: parentV];
    
    [window addSubview: viewController.view];
    [window makeKeyAndVisible];
    
    if (YES == ret) {
        [view onCreate];
    }
    
    return ret;
}

- Kim

Re: Playing Video Over the image

August 26, 2011 - 11:09am #17

can somebody help me with a sample example....

or tell me where I have to write the code

viewController = [[ARViewController alloc] init];
parentView = [[UIView alloc] initWithFrame: screenBounds];
newView = [[EAGLView alloc] initWithFrame: screenBounds];
[parentView addSubview:view];
NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"teaser" ofType:@"mp4"]];
video = [[MPMoviePlayerController alloc]initWithContentURL:url]];
[video.view setFrame:CGRectMake(50, 100, 500, 500)];
[video play];
[parentView addSubview:video.view];
[viewController setView: parentV];

in the ImageTargetsAppDelegate

or in EAGLView

Re: Playing Video Over the image

August 25, 2011 - 3:34am #16

Hi nm,

It would be worth checking that the NSURL and 'video' are valid objects.

The path you are giving should either have its full path added or, if the video is in the bundle, you should be using something like:

NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"Senna" ofType:@"mov"]];

Also the example I gave had an intermediate view 'videoV' that I added the video view to. It may be worth adding that indirection if the above checks don't reveal anything.

Re: Playing Video Over the image

August 25, 2011 - 2:30am #15

Thanks for reply. Yes i had tested my video and its working fine other application.

As u said in your earlier post to create a parent view which parents both EAGLEview and video view the code is as follows:

viewController = [[ARViewController alloc] init];
parentView = [[UIView alloc] initWithFrame: screenBounds];
newView = [[EAGLView alloc] initWithFrame: screenBounds];
[parentView addSubview:view];
video = [[MPMoviePlayerController alloc]initWithContentURL:[NSURL fileURLWithPath:@"teaser.mp4"]];
[video.view setFrame:CGRectMake(50, 100, 500, 500)];
[video play];
[parentView addSubview:video.view];
[viewController setView: parentV];

plz tel me where i m going wrong??

Re: Playing Video Over the image

August 23, 2011 - 9:50am #14
nm wrote:

I am also trying to play video over image but i am also getting the same problem as yours method is called but video is not playing.

You've probably done this, but check the the video runs in the player view in a test app before you start integrating with AR. When I was testing there were quite a few sample videos that wouldn't run on Apple's video player in iOS. I finally used the Ayrton Senna movie trailer for my tests which worked fine.

Re: Playing Video Over the image

August 23, 2011 - 3:59am #13

Hi shivintu,

I am also trying to play video over image but i am also getting the same problem as yours method is called but video is not playing.

Can u plz help me with this?

Re: Playing Video Over the image

August 9, 2011 - 11:54pm #12
shivintu wrote:

but i am unable to change the rect of "videoV" using "s0,s1,s2,s3" Could you please explain these parameters and how can we give x,y,width,height to the videoV rect.

As I stated earlier: "Using the max/min in each dimension will allow you to specify another UIView that hides the target"

I'm in danger of writing your app for you here ;), but in pseudo-code:

x = minVal(s0.x,s1.x,s2.x,s3.x);
width = maxVal(s0.x,s1.x,s2.x,s3.x) - x;

etc. You may want to adjust the final result based on the aspect ratio of the video being played.

Re: Playing Video Over the image

August 9, 2011 - 10:58pm #11
MoSR wrote:

BTW - the target size in 3D units, that I've hard coded as {247,173}, can be recovered from the Trackable, by using it's subclass, eg. ImageTarget, see:

QCAR::ImageTarget::getSize()

Hi MoSR,

Thank you for posting the sample code, i tried it and even as you said i am able to get the image size using the below code and i updated CGSize target = {247,173} with dynamic sizes.

const QCAR::ImageTarget* imageTarget = static_cast(trackable);
QCAR::Vec2F::Vec2F screenPoint = imageTarget->getSize();
CGSize target;
target.width = screenPoint.data[0];
target.height = screenPoint.data[1];
// CGSize target = {247,173};
[self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

but i am unable to change the rect of "videoV" using "s0,s1,s2,s3" Could you please explain these parameters and how can we give x,y,width,height to the videoV rect.

Re: Playing Video Over the image

August 9, 2011 - 11:48am #10

did you have any luck with it Shivintu?

Re: Playing Video Over the image

August 9, 2011 - 10:27am #9

BTW - the target size in 3D units, that I've hard coded as {247,173}, can be recovered from the Trackable, by using it's subclass, eg. ImageTarget, see:

QCAR::ImageTarget::getSize()

Re: Playing Video Over the image

August 9, 2011 - 10:15am #8

The following will give you the four coordinates of the target corners within the EAGLView. Using the max/min in each dimension will allow you to specify another UIView that hides the target.

I used this code to display a different coloured 10x10-pixel UIView at each corner of the target. Note the adjustment for the camera view being deeper than the screen (the screen clips the camera view - this may explain why your earlier attempt was offset).

s0 thro' s3 are new public properties of type CGPoint, in EAGLView. Your delegate code can access this to reposition other views.

- (CGPoint) projectCoord:(CGPoint)coord inView:(const QCAR::CameraCalibration&)cameraCalibration andPose:(QCAR::Matrix34F)pose withOffset:(CGPoint)offset
{
    CGPoint converted;
    
    QCAR::Vec3F vec(coord.x,coord.y,0);
    QCAR::Vec2F sc = QCAR::Tool::projectPoint(cameraCalibration, pose, vec);
    converted.x = sc.data[0] - offset.x;
    converted.y = sc.data[1] - offset.y;
    
    return converted;
}

- (void) calcScreenCoordsOf:(CGSize)target inView:(CGFloat *)matrix inPose:(QCAR::Matrix34F)pose
{
    // 0,0 is at centre of target so extremities are at w/2,h/2
    CGFloat w = target.width/2;
    CGFloat h = target.height/2;
    
    const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
    const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration();

    // calculate any mismatch of screen to video size
    QCAR::CameraDevice& cameraDevice = QCAR::CameraDevice::getInstance();
    QCAR::VideoMode videoMode = cameraDevice.getVideoMode(QCAR::CameraDevice::MODE_DEFAULT);
    CGPoint margin = {(videoMode.mWidth - self.frame.size.width)/2, (videoMode.mHeight - self.frame.size.height)/2};

    // now project the 4 corners of the target
    s0 = [self projectCoord:CGPointMake(-w,h) inView:cameraCalibration andPose:pose withOffset:margin];
    s1 = [self projectCoord:CGPointMake(-w,-h) inView:cameraCalibration andPose:pose withOffset:margin];
    s2 = [self projectCoord:CGPointMake(w,-h) inView:cameraCalibration andPose:pose withOffset:margin];
    s3 = [self projectCoord:CGPointMake(w,h) inView:cameraCalibration andPose:pose withOffset:margin];
}

----8<---- in renderFrameQCAR ----8<----

ShaderUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale, &modelViewMatrix.data[0]);
ShaderUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale, &modelViewMatrix.data[0]);
ShaderUtils::multiplyMatrix(&projectionMatrix.data[0], &modelViewMatrix.data[0], &modelViewProjection.data[0]);
            
 CGSize target = {247,173};
[self calcScreenCoordsOf:target inView:&modelViewProjection.data[0] inPose:trackable->getPose()];

Re: Playing Video Over the image

August 9, 2011 - 8:37am #7
MoSR wrote:

When you create the trackable, you specify a size of the trackable in your own 'internal units' (see config.xml in the project) - the scale information from the pose is relative to that.

For example if your trackable is 50x50 units then if you take a 50 by 50 quad and transform it by the projection matrix and the trackable pose, and then project those points to screen, it should give you the screen coordinates of the target.

Well that's the theory anyway - I'll try this as soon as I can and post the code as an example.

Ok, That would be really helpful for me. I will also try. Thank you for spending valuable time.

Re: Playing Video Over the image

August 9, 2011 - 6:45am #6
shivintu wrote:

I want to place the video screen exactly on top of the traced image. i.e even i move camera close to the image or far to the image, just video player should be placed on top it.

When you create the trackable, you specify a size of the trackable in your own 'internal units' (see config.xml in the project) - the scale information from the pose is relative to that.

For example if your trackable is 50x50 units then if you take a 50 by 50 quad and transform it by the projection matrix and the trackable pose, and then project those points to screen, it should give you the screen coordinates of the target.

Well that's the theory anyway - I'll try this as soon as I can and post the code as an example.

Re: Playing Video Over the image

August 9, 2011 - 5:47am #5

[QUOTE=MoSR" />http://ar.qualcomm.at/node/2000523How to export target coordinates to java?

2) This may be the latency of the video player redraw - try drawing a small coloured view instead and see how it tracks.[/QUOTE" />

Hi MoSR, Thank you for reply.

The code which you have provided in the attached link is I already used it, and i am able to get the x,y positions but it seems those are pointing to the center of target.

I want to place the video screen exactly on top of the traced image. i.e even i move camera close to the image or far to the image, just video player should be placed on top it.

Please help me on this, i am trying for that.

Re: Playing Video Over the image

August 9, 2011 - 4:05am #4

[QUOTE=shivintu" />http://ar.qualcomm.at/node/2000523>

1) Using QCAR::Image won't give you what you need as what you want is the projected size of the image in the view, not the original size of the source image. Use the 'projectionMatrix' and QCAR::Trackable::getPose() to determine the trackable target's location and orientation. See the thread:

How to export target coordinates to java?

2) This may be the latency of the video player redraw - try drawing a small coloured view instead and see how it tracks.

Re: Playing Video Over the image

August 8, 2011 - 7:44am #3
MoSR wrote:

Hi shivintu,

To add additional UI, whether video views or UI controls, it's probably best practise to create a new parent view, that parents both the EAGLView and a view that contains the overlaid content.

I've done that here as a test, to play the video in a 2D pop-up overlaying the 3D/Camera view and it works fine.

Remember that the UIScreen bounds will not be in the same orientation as the sub-views in the ImageTargets case as they are forced landscape, so set the EAGLView subview frame to be [0,0,screenBounds.size.height, screenBounds.size.width].

CGRect screenBounds = [[UIScreen mainScreen] bounds];

window = [[UIWindow alloc] initWithFrame: screenBounds];
viewController = [[ARViewController alloc] init];
parentV = [[UIView alloc] initWithFrame: screenBounds];

screenBounds = CGRectMake(0, 0, parentV.frame.size.height, parentV.frame.size.width);
view = [[EAGLView alloc] initWithFrame: screenBounds];
[parentV addSubview:view];

CGRect subScreenBounds = CGRectMake(500, 100, 500, 500);
videoV = [[UIView alloc] initWithFrame:subScreenBounds];
[parentV addSubview:videoV];

[viewController setView: parentV];

Note that to play video within the 3D world you'd need to sync the video frame rate with the camera frame rate and insert each frame into the 3D view as the texture of a displayed object. This is unlikely to yield good performance as you are bypassing all the optimisation in the movie player.

Thank you very much MoSR, I tried it and its working, I have another problem. How we can place the video screen exactly on top of the traced image area with same width and height of traced image. and also as image is scaling (if i move the camera close to the image then phone entire screen will be occupied by image and when i am going far to the image then occupation of image on mobile real estate will be reduced) how can we scale the video screen.

I have tired below code to place the video screen on top of traced image.

const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration();
QCAR::Vec2F screenPoint = QCAR::Tool::projectPoint(cameraCalibration, trackable->getPose(), QCAR::Vec3F(0, 0, 0));
targetXPos = screenPoint.data[0];
targetYPos = screenPoint.data[1];
videoV setFrame:cgrectMake(targetXpos,targetYPos,237,150);

Here i am give width and height are static, bcz i am not able to get image width and height. i have tried QCAR::Image::getWidth(); API but as there is no static function to get the Image class instance, not getting to get the image width and height dynamically.

The video screen is not placing exactly on top of image. but its moving along with image with some rect difference as image x and y pos's are traced out.

I need help on how we can place video screen exactly on top of traced image with same width and height of traced image as well how we can scale it.

Please help me.

Re: Playing Video Over the image

August 8, 2011 - 3:34am #2

Hi shivintu,

To add additional UI, whether video views or UI controls, it's probably best practise to create a new parent view, that parents both the EAGLView and a view that contains the overlaid content.

I've done that here as a test, to play the video in a 2D pop-up overlaying the 3D/Camera view and it works fine.

Remember that the UIScreen bounds will not be in the same orientation as the sub-views in the ImageTargets case as they are forced landscape, so set the EAGLView subview frame to be [0,0,screenBounds.size.height, screenBounds.size.width].

CGRect screenBounds = [[UIScreen mainScreen] bounds];

window = [[UIWindow alloc] initWithFrame: screenBounds];
viewController = [[ARViewController alloc] init];
parentV = [[UIView alloc] initWithFrame: screenBounds];

screenBounds = CGRectMake(0, 0, parentV.frame.size.height, parentV.frame.size.width);
view = [[EAGLView alloc] initWithFrame: screenBounds];
[parentV addSubview:view];

CGRect subScreenBounds = CGRectMake(500, 100, 500, 500);
videoV = [[UIView alloc] initWithFrame:subScreenBounds];
[parentV addSubview:videoV];

[viewController setView: parentV];

Note that to play video within the 3D world you'd need to sync the video frame rate with the camera frame rate and insert each frame into the 3D view as the texture of a displayed object. This is unlikely to yield good performance as you are bypassing all the optimisation in the movie player.

Pages

Log in or register to post comments