Log in or register to post comments

3D touch error

July 17, 2013 - 11:09am #1

Hi,

I have found the way to transform touches on the screen, to points in 3D space following the Dominoes sample application.

My trouble is that the transformed touches are "translated" to the left by some points leading to incorrect results. I have followed the code in the dominoes app and there are no differences with mine, the projection matrix is also the same. I had concerns that I accidentally modified the modelview matrix before using it to transform 2D to 3D points but this is not the case.

Any ideas why the points of intersection with the marker plane are offset to the left?

Thanks,

Mike

3D touch error

September 27, 2013 - 12:16am #14

Your welcome!

Glad I could help.

 

N

3D touch error

September 26, 2013 - 7:17am #13

Nalin,

I can't thank you enough for your last answer! Although it was not definitive it did point me to the right direction.

In my ParentViewController I was handling the rotation for the controller itself instead of the ARViewController. In addition, and because I wanted to enable gestures in my OverlayViewController, I handled all touches within that controller's view instead of the EAGLView directly. This is why my coordinate system orientation did not always 'follow' the device orientation changes and I had to manually adjust it.

By receiving touches from the EAGLView directly, everything works as it should!

I only wish I had asked the right questions earlier. Anyway, thanks again.

M.

3D touch error

September 25, 2013 - 3:48am #12

Sorry this is causing you such problems.

Both applications are using the same ARCommon Directory as a base and have a similar structure.  Given this you can plot any differences by going through the file groups  in xcode:  ImageTargets/Classes/ImageTargets  v Dominoes.

One area where these are different is in the ParentView Controller as the Dominoes implementation seems to override the default handleARViewRotation, though I cannot say if this is having any effect.

 

N

 

3D touch error

September 24, 2013 - 10:50am #11

Having modified yet another ImageTargets sample I found out that when the device is rotated in landscape mode, the coordinate system of the image target is rotated by 180 degrees. This is why in my case, the intersections with the target plane were going haywire.

In addition to the above, touching the iphone screen on the farther edge while pointing away from the marker will register a correct point in the marker coordinate system of the Dominoes sample. That point could be in the range (-400, 400) in the x axis and (-300, 800) in the y axis. If the same is tried with the ImageTargets sample the tranformed point will "fold" into a coordinate system ranging from (-200, 200) in the x axis and (-120, 120) in the y. This has a devastating effect on the accuracy of picking objects.

This does not happen in the Dominoes demo. Does anyone know why that is? What is the difference in QCAR setup in the 2 demos and they behave so differently? Putting handleTouches from the Dominoes sample into the ImageTargets app does certainly not mean one can have a working picking behaviour in the latter.

I having been looking into this for over a month and a half and would appreciate it if someone would give me an answer on the matter.

 

 

3D touch error

September 18, 2013 - 1:27am #10

Victor,

I suspect that this is a coordinate mapping issue. When I checked the dominoes sample for intersection points the x-y axes seemed reversed. I will have to have a more thorough look into this but for now I think that the coordinate system used in the image targets sample is different. If I manage to solve this I will let you know.

M.

3D touch error

September 4, 2013 - 9:21pm #9

Not an answer , but I am facing the same issue , the touch points are translated to bottom left if I am close to the target image and top right if i take the camera far away from the target image. Somewhere in between they seem to be aligned properly. Any clue why this may be happening ?

Thanks

3D touch error

September 4, 2013 - 5:13am #8

I am using the standard "stones" image target by Vuforia both for the Dominoes and the ImageTargets samples. Is there a setup for unit changes within the xcode ImageTargets sample?

As a different solution, I thought of projecting the minimum and maximum points of the bounding box of each 3d object in the scene into screen space; and then checking if the point of touch is within the rectangle specified by that pair. To that end, I have used the following code:

bool objectBoundingBoxContainsTouchPoint(REVuModel *cameraObject, QCAR::Vec2F &point){
    QCAR::Vec3F minVector, maxVector;
    cameraObject->GetBoundingBox(minVector, maxVector);
    
    // Project the 2 vectors in screen space
    QCAR::Matrix44F modelViewProjection = QCAR::Tool::multiply(qUtils->modelViewMatrix, qUtils->projectionMatrix);
    QCAR::Vec3F screenMin = SampleMath::Vec3FTransform(minVector, modelViewProjection);
    QCAR::Vec3F screenMax = SampleMath::Vec3FTransform(maxVector, modelViewProjection);
    
    QCAR::VideoBackgroundConfig config = QCAR::Renderer::getInstance().getVideoBackgroundConfig();
    
    float halfScreenWidth = qUtils.viewSize.height / 2.0f;  // inversed - matches width of the target
    float halfScreenHeight = qUtils.viewSize.width / 2.0f; // inversed - matches height of the target
    
    float halfViewportWidth = config.mSize.data[0] / 2.0f; 
    float halfViewportHeight = config.mSize.data[1] / 2.0f;
    
    float sMinX = (screenMin.data[0] * halfViewportWidth + halfScreenWidth) / qUtils.contentScalingFactor;
    float sMinY = (screenMin.data[1] * halfViewportHeight + halfScreenHeight) / qUtils.contentScalingFactor;
    
    printf("Min vec: %.2f, %.2f\n", sMinX, sMinY);
    
    float sMaxX = (screenMax.data[0] * halfViewportWidth + halfScreenWidth) / qUtils.contentScalingFactor;
    float sMaxY = (screenMax.data[1] * halfViewportHeight + halfScreenHeight) / qUtils.contentScalingFactor;
    
    printf("Max vec: %.2f, %.2f\n", sMaxX, sMaxY);
    printf("Touch: %.2f, %.2f", point.data[0] , point.data[1] );

    return ((point.data[0] > sMinX && point.data[0] < sMaxX) && (point.data[1] > sMinY && point.data[1] < sMaxY));
    
}

So far it has not worked. I might be doing something wrong with the transformations. What do you think?

Thanks,

M.

3D touch error

September 3, 2013 - 9:46am #7

I am not able to test out the code at this moment.

One possibility I could think of is the Scene Units as specified in the Target Manager.  It might be worth experimenting a little here with different targets, each with different units.

 

N

3D touch error

September 3, 2013 - 9:29am #6

Nalin,

thanks for your reply.

A colleague and myself both substituted the HandleTouches code in the Dominoes app with our version and it (the Dominoes app) kept finding the touch points correctly. We are still puzzled by what could the problem be. Would you be so kind to try and replicate the issue by modifying the ImageTargets sample, using the steps I mentioned in my previous post? I know that other people are having problems with this too (https://developer.vuforia.com/forum/ios/getting-touch-event-3d-model and https://developer.vuforia.com/forum/ios/intersection-point-target-overlay-cloud-reco-example) as the Dominoes sample is differently laid out compared to the ImageTargets sample.

Otherwise, could this be an app configuration issue, i.e. could the two samples have different tracking configurations and the like, to cause different behaviour?

Cheers,

M.

3D touch error

July 25, 2013 - 4:12am #5

Nothing obvious springs to mind from looking at the code, but it is hard to debug.

If I understand correctly what you are saying is that the touch point is fine in the dominoes app, but your version in your own app results in it being skewed to the left.

 

If I were to try and narrow this down, I would use your new version in the dominoes app as a straight substitution to identify where the problem is - either in your version of the touch code, or something about the app itself.

 

There has to be a reason behind this - so this is what I would suggest to help identify where it might be.

 

N

3D touch error

July 24, 2013 - 11:09am #4

I have made another project modifying the ImageTargets sample to make sure I was doing things the right way.

What I have noticed is that although lineStart seems correct, the lineEnd point is skewed by a great amount to the left. Why would that be?

Could someone please help me out with this? I have been struggling with it for 2 weeks and I am beginning to pull my hair out.

You could replicate my steps by modifying the ImageTargets sample project.

I have added the following method to QCARUtils.mm :

 

bool touchPointIntersectsCameraObject(QCAR::Vec2F &point)
{
    QCARutils *qUtils = [QCARutils getInstance];
    
    // Window Coordinates to Normalized Device Coordinates
    QCAR::VideoBackgroundConfig config = QCAR::Renderer::getInstance().getVideoBackgroundConfig();
    
    float halfScreenWidth = qUtils.viewSize.height / 2.0f;  // inversed - matches width of the target
    float halfScreenHeight = qUtils.viewSize.width / 2.0f; // inversed - matches height of the target
    
    float halfViewportWidth = config.mSize.data[0] / 2.0f;
    float halfViewportHeight = config.mSize.data[1] / 2.0f;
    
    float x = (qUtils.contentScalingFactor * point.data[0] - halfScreenWidth) / halfViewportWidth;
    float y = (qUtils.contentScalingFactor * point.data[1] - halfScreenHeight) / halfViewportHeight * -1;
    
    QCAR::Vec4F ndcNear(x, y, -1, 1);
    QCAR::Vec4F ndcFar(x, y, 1, 1);
    
    // Normalized Device Coordinates to Eye Coordinates
    QCAR::Matrix44F projectionMatrix = [QCARutils getInstance].projectionMatrix;
    QCAR::Matrix44F inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
    
    QCAR::Vec4F pointOnNearPlane = SampleMath::Vec4FTransform(ndcNear, inverseProjMatrix);
    QCAR::Vec4F pointOnFarPlane = SampleMath::Vec4FTransform(ndcFar, inverseProjMatrix);
    pointOnNearPlane = SampleMath::Vec4FDiv(pointOnNearPlane, pointOnNearPlane.data[3]);
    pointOnFarPlane = SampleMath::Vec4FDiv(pointOnFarPlane, pointOnFarPlane.data[3]);
    
    // Eye Coordinates to Object Coordinates
    QCAR::Matrix44F modelViewMatrix = [QCARutils getInstance].modelViewMatrix;
    
    QCAR::Matrix44F inverseModelViewMatrix = SampleMath::Matrix44FInverse(modelViewMatrix);
    
    QCAR::Vec4F nearWorld = SampleMath::Vec4FTransform(pointOnNearPlane, inverseModelViewMatrix);
    QCAR::Vec4F farWorld = SampleMath::Vec4FTransform(pointOnFarPlane, inverseModelViewMatrix);
    
    QCAR::Vec3F lineStart = QCAR::Vec3F(nearWorld.data[0], nearWorld.data[1], nearWorld.data[2]);
    QCAR::Vec3F lineEnd = QCAR::Vec3F(farWorld.data[0], farWorld.data[1], farWorld.data[2]);
    
//    NSLog(@"Line start: %.2f, %.2f, %.2f", lineStart.data[0], lineStart.data[1], lineStart.data[2]);
//    NSLog(@"Line end: %.2f, %.2f, %.2f", lineEnd.data[0], lineEnd.data[1], lineEnd.data[2]);
   

if (linePlaneIntersection(lineStart, lineEnd, QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection)) {
        NSLog(@"Intersection is: %.2f, %.2f, %.2f", intersection.data[0], intersection.data[1], intersection.data[2]);
        [EAGLView addTouchPointX:intersection.data[0] Y:intersection.data[1] Z:intersection.data[2]];
    }
 
    return false;
}

which is almost the same as in the Dominoes sample. I obtain the modelview matrix using:

 

modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(result->getPose());

 

within renderFrameQCAR, which I later on pass in to the touchPointIntersectsCameraObject method through a member var in QCARUtils. I make sure that the modelview is not modified along the way.

I have added a touchesBegan method within ImageTargetsOverlayViewController that goes like:

 

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event{
    UITouch *touch = [touches anyObject];
    CGPoint pointOfTouch =  [touch locationInView:self.view];
    
    BOOL touchedVirtualObject = [[QCARutils getInstance] didTouchVirtualObjectAtPoint:pointOfTouch];
    if (touchedVirtualObject) {
        NSLog(@"You touched a virtual object!");
    }
}

and set UserInteractionEnabled to YES for the ImageTargetsOverlayViewController to receive touches.

What else could I be missing?

Many thanks,

Mike

3D touch error

July 18, 2013 - 1:44am #3

No.

I am wondering if the following code is sufficient for my needs:

    QCARutils *qUtils = [QCARutils getInstance];
    
    // Window Coordinates to Normalized Device Coordinates
    QCAR::VideoBackgroundConfig config = QCAR::Renderer::getInstance().getVideoBackgroundConfig();
    
    float halfScreenWidth = qUtils.viewSize.height / 2.0f;  // inversed - matches width of the target
    float halfScreenHeight = qUtils.viewSize.width / 2.0f; // inversed - matches height of the target
    
    float halfViewportWidth = config.mSize.data[0] / 2.0f;
    float halfViewportHeight = config.mSize.data[1] / 2.0f;
    
    float x = (qUtils.contentScalingFactor * point.data[0] - halfScreenWidth) / halfViewportWidth;
    float y = (qUtils.contentScalingFactor * point.data[1] - halfScreenHeight) / halfViewportHeight * -1;
    
    QCAR::Vec4F ndcNear(x, y, -1, 1);
    QCAR::Vec4F ndcFar(x, y, 1, 1);
    
    // Normalized Device Coordinates to Eye Coordinates
    QCAR::Matrix44F projectionMatrix = [QCARutils getInstance].projectionMatrix;
    QCAR::Matrix44F inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
    
    QCAR::Vec4F pointOnNearPlane = SampleMath::Vec4FTransform(ndcNear, inverseProjMatrix);
    QCAR::Vec4F pointOnFarPlane = SampleMath::Vec4FTransform(ndcFar, inverseProjMatrix);
    pointOnNearPlane = SampleMath::Vec4FDiv(pointOnNearPlane, pointOnNearPlane.data[3]);
    pointOnFarPlane = SampleMath::Vec4FDiv(pointOnFarPlane, pointOnFarPlane.data[3]);
    
    // Eye Coordinates to Object Coordinates
    QCAR::Matrix44F modelViewMatrix = [QCARutils getInstance].modelViewMatrix;
    QCAR::Matrix44F inverseModelViewMatrix = SampleMath::Matrix44FInverse(modelViewMatrix);

    QCAR::Vec4F nearWorld = SampleMath::Vec4FTransform(pointOnNearPlane, inverseModelViewMatrix);
    QCAR::Vec4F farWorld = SampleMath::Vec4FTransform(pointOnFarPlane, inverseModelViewMatrix);
    
    QCAR::Vec3F lineStart = QCAR::Vec3F(nearWorld.data[0], nearWorld.data[1], nearWorld.data[2]);
    QCAR::Vec3F lineEnd = QCAR::Vec3F(farWorld.data[0], farWorld.data[1], farWorld.data[2]);

    QCAR::Vec3F intersection;

    bool intersectsPlane = linePlaneIntersection(lineStart, lineEnd, QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection);
    if (intersectsPlane) {
        NSLog(@"Intersection is: %.2f, %.2f, %.2f", intersection.data[0], intersection.data[1], intersection.data[2]);
    }

    return intersectsPlane;

Or if I should call updatePickingTransform on camera view change.

M.

3D touch error

July 18, 2013 - 1:24am #2

Are the touches translated left in the Dominoes sample?

 

N

Log in or register to post comments