"We offer new support options and therefor the forums are now in read-only mode! Please check out our Support Center for more information." - Vuforia Engine Team

2d tap detection

My app is based on Video Playback (but it's already modified a lot). I'm implementing buttons on video (which are simple textures). I'm rendering buttons (textures) on X, Y and Z axis and it works as expected.

 

I have a problem with detecting tap action on those buttons. Based on tap detection that is done in Video Playback I did something similar.

 

Video Playback uses method "projectScreenPointToPlane" and then compares "intersection" with half of the size of trackable element for each side (making 3d problem a 2d problem). However, I think that it doesn't work correctly. Video targets are quite big so it's hard to debug but I noticed the problem when I created smaller tappable areas which are buttons.

 

As far as I understand intersection from method "projectScreenPointToPlane" should give X and Y from the CENTER of trackable right? From my testing X and Y values change a lot depending on the camera angle and trackable position. If I move my trackable to the center of screen than values are correct.

 

This is easy to reproduce (debug). Just put "NSLog(@"x: %f  y: %f", intersection.data[0], intersection.data[1]);" in method "tapInsideTargetWithID" in file EAGLView.mm. Open the app, move the camera to different angles and try to tap on a center of the trackable. As far as I understand it should give you values near to 0 (if you tap in the center) but it gives different values based on camera position.

 

I'm I doing something wrong and "projectScreenPointToPlane" works in a different way?

 

// Determine whether a screen tap is inside the target
- (int)tapInsideTargetWithID
{
    QCAR::Vec3F intersection, lineStart, lineEnd;
    // Get the current projection matrix
    QCAR::Matrix44F projectionMatrix = [[QCARControl getInstance] projectionMatrix];
    QCAR::Matrix44F inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
    CGRect rect = [self bounds];
    int touchInTarget = -1;
    
    // ----- Synchronise data access -----
    [dataLock lock];
    
    // The target returns as pose the centre of the trackable.  Thus its
    // dimensions go from -width / 2 to width / 2 and from -height / 2 to
    // height / 2.  The following if statement simply checks that the tap is
    // within this range
    for (int i = 0; i < NUM_VIDEO_TARGETS; ++i) {
        SampleMath::projectScreenPointToPlane(inverseProjMatrix, videoData[i].modelViewMatrix, rect.size.width, rect.size.height,
                                              QCAR::Vec2F(touchLocation_X, touchLocation_Y), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);
        
        
        NSLog(@"x: %f  y: %f", intersection.data[0], intersection.data[1]);
        
        if ((intersection.data[0] >= -videoData[i].targetPositiveDimensions.data[0]) && (intersection.data[0] <= videoData[i].targetPositiveDimensions.data[0]) &&
            (intersection.data[1] >= -videoData[i].targetPositiveDimensions.data[1]) && (intersection.data[1] <= videoData[i].targetPositiveDimensions.data[1])) {
            // The tap is only valid if it is inside an active target
            if (YES == videoData[i].isActive) {
                touchInTarget = i;
                break;
            }
        }
    }

    [dataLock unlock];
    // ----- End synchronise data access -----
    
    return touchInTarget;
}

 

From my testing X and Y values change a lot depending on the camera angle and trackable position

Have you tested this functionin the Dominoes sample?  

If so how is it different from this?  

The Dominoes sample also uses projectScreenPointToPlane() in the handleTouches() method in Dominoes.mm so this would be a good starting point to experiment and learn. 

Thanks for your help. You were right about the method "projectScreenPointToPlane", it is simply messed in the VideoPlayback sample.

I just noticed that the newest source code in VideoPlayback uses something like QCARControl istead of QCARUtils. Here is the code for QCARUtils used like in Dominos sample.

 

Method "tapInsideTargetWithID" in EAGLView.m

Thanks for this.

Can I ask what problems were you seeing with the existing code in the existing video playback sample?

You say the code was a bit messed up, but what was the impact?

Also, presumably you made changes to both the methods supplied?

 

cheers

The impact is that method "projectScreenPointToPlane" was not giving intersection X and Y from the center of the trackable, it was dependend on the came

I only used this with QCARUtils (which is used across most of the samples) in version 2.0.7. Version 2.6.6. of VideoPlayback uses something different but no problem I will track the problem tomorrow and send you code for 2.6.6 :)