I only used this with QCARUtils (which is used across most of the samples) in version 2.0.7. Version 2.6.6. of VideoPlayback uses something different but no problem I will track the problem tomorrow and send you code for 2.6.6 :)

I am getting errors on these lines:

    CGSize size = [[QCARControl getInstance] viewSize];

    CGFloat contentScalingFactor = [[QCARControl getInstance] contentScalingFactor];

did you define methods for these that I can hang off QCARControl to try this out?

N

The impact is that method "projectScreenPointToPlane" was not giving intersection X and Y from the center of the trackable, it was dependend on the camera position. See my first post for more explanation. Photo that I attached also illustrates this issue:

https://developer.vuforia.com/sites/default/files/sample.jpg

If the method "projectScreenPointToPlane" would work correctly X on the center should be near 0.0 and it should be the same on the left and on the right (of course one should be "-" and one "+"). I think correct value for this trackable is around -60 for left and +60 for right.

Thanks for this.

Can I ask what problems were you seeing with the existing code in the existing video playback sample?

You say the code was a bit messed up, but what was the impact?

Also, presumably you made changes to both the methods supplied?

cheers

N

I just noticed that the newest source code in VideoPlayback uses something like QCARControl istead of QCARUtils. Here is the code for QCARUtils used like in Dominos sample.

Method "tapInsideTargetWithID" in EAGLView.m

- (int)tapInsideTargetWithID
{
    QCAR::Vec3F intersection, lineStart, lineEnd;
    QCAR::Matrix44F projectionMatrix = [QCARutils getInstance].projectionMatrix;
    QCAR::Matrix44F inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
    CGSize size = [[QCARutils getInstance] viewSize];
    CGFloat contentScalingFactor = [[QCARutils getInstance] contentScalingFactor];
    int touchInTarget = -1;
    
    // ----- Synchronise data access -----
    [dataLock lock];
    
    // The target returns as pose the centre of the trackable.  Thus its
    // dimensions go from -width / 2 to width / 2 and from -height / 2 to
    // height / 2.  The following if statement simply checks that the tap is
    // within this range
    for (int i = 0; i < NUM_VIDEO_TARGETS; ++i) {
        SampleMath::projectScreenPointToPlane(inverseProjMatrix, videoData[i].modelViewMatrix, contentScalingFactor, size.width, size.height,
                                              QCAR::Vec2F(touchLocation_X, touchLocation_Y), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);
        
        if ((intersection.data[0] >= -videoData[i].targetPositiveDimensions.data[0]) && (intersection.data[0] <= videoData[i].targetPositiveDimensions.data[0]) &&
            (intersection.data[1] >= -videoData[i].targetPositiveDimensions.data[1]) && (intersection.data[1] <= videoData[i].targetPositiveDimensions.data[1])) {
            // The tap is only valid if it is inside an active target
            if (YES == videoData[i].isActive) {
                touchInTarget = i;
                break;
            }
        }
    }
    
    [dataLock unlock];
    // ----- End synchronise data access -----
    
    return touchInTarget;
}

Method "projectScreenPointToPlane" in SampleMath

SampleMath::projectScreenPointToPlane(QCAR::Matrix44F inverseProjMatrix, QCAR::Matrix44F modelViewMatrix,
                          float contentScalingFactor, float screenWidth, float screenHeight,
                          QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal,
                          QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd)
{
    // Window Coordinates to Normalized Device Coordinates
    QCAR::VideoBackgroundConfig config = QCAR::Renderer::getInstance().getVideoBackgroundConfig();
    
    float halfScreenWidth = screenHeight / 2.0f;
    float halfScreenHeight = screenWidth / 2.0f;
    
    float halfViewportWidth = config.mSize.data[0] / 2.0f;
    float halfViewportHeight = config.mSize.data[1] / 2.0f;
    
    float x = (contentScalingFactor * point.data[0] - halfScreenWidth) / halfViewportWidth;
    float y = (contentScalingFactor * point.data[1] - halfScreenHeight) / halfViewportHeight * -1;
    
    QCAR::Vec4F ndcNear(x, y, -1, 1);
    QCAR::Vec4F ndcFar(x, y, 1, 1);
    
    // Normalized Device Coordinates to Eye Coordinates
    QCAR::Vec4F pointOnNearPlane = Vec4FTransform(ndcNear, inverseProjMatrix);
    QCAR::Vec4F pointOnFarPlane = Vec4FTransform(ndcFar, inverseProjMatrix);
    pointOnNearPlane = Vec4FDiv(pointOnNearPlane, pointOnNearPlane.data[3]);
    pointOnFarPlane = Vec4FDiv(pointOnFarPlane, pointOnFarPlane.data[3]);
    
    // Eye Coordinates to Object Coordinates
    QCAR::Matrix44F inverseModelViewMatrix = Matrix44FInverse(modelViewMatrix);
        
    QCAR::Vec4F nearWorld = Vec4FTransform(pointOnNearPlane, inverseModelViewMatrix);
    QCAR::Vec4F farWorld = Vec4FTransform(pointOnFarPlane, inverseModelViewMatrix);
        
    lineStart = QCAR::Vec3F(nearWorld.data[0], nearWorld.data[1], nearWorld.data[2]);
    lineEnd = QCAR::Vec3F(farWorld.data[0], farWorld.data[1], farWorld.data[2]);
    linePlaneIntersection(lineStart, lineEnd, planeCenter, planeNormal, intersection);
}

Please by all means post this up and I will try to take a look :)

N

I tried to edit previous post but it did not work :)

I do not know what is your policy on fixing sample code but I can push some fixed code for review if you want to (I guess it's not on github or something?). If you manage it internally a ticket with reference for this thread should be more than enough to fix it.

Great to hear it's working for you :)

N

Thanks for your help. You were right about the method "projectScreenPointToPlane", it is simply messed in the VideoPlayback sample. I compared it line by line with Dominos and in VideoPlayback it takes wrong size parameters. It should use size from [[QCARUtilis getinstance] viewSize] and include [[QCARUtils getInstance] contentScalingFactor] in the calculations. I just don't know why in Dominos heigh and width is switched (see code below) but it seem to work.

float halfScreenWidth = qUtils.viewSize.height / 2.0f; // note use of height for width
float halfScreenHeight = qUtils.viewSize.width / 2.0f; // likewise

The Dominoes sample also uses projectScreenPointToPlane() in the handleTouches() method in Dominoes.mm so this would be a good starting point to experiment and learn. 

Within this method it also uses  checkIntersectionLine()  to check whether the line from  projectScreenPointToPlane()  intersects with the domino:

            bool intersection = checkIntersectionLine(domino->pickingTransform, lineStart, lineEnd);

There is also further discussion about this topic in this thread:

https://developer.vuforia.com/forum/ios/getting-touch-event-3d-model

Most of what you want to do is already there, so you should be able to repurpose this without too much trouble.

HTH

N

Have you tested this functionin the Dominoes sample?  

As far as I know dominos uses 3d tap detection by using "checkIntersectionLine". Can I also make something similar here?

Do the X and Y values change as much, more or less in this sample compared to yours?

Yes mostly the same. Of course it depends on the trackable size but it's the same. I thought I did something wrong that's why I opened clean Video Playback sample and tested it there. See attached photo to see what I mean.

I think it's something with the Z axis.

BTW. Can I do tap detection on textures with "checkIntersectionLine" like in Dominos sample?

From my testing X and Y values change a lot depending on the camera angle and trackable position

Have you tested this functionin the Dominoes sample?  

If so how is it different from this?  

Do the X and Y values change as much, more or less in this sample compared to yours?

N