My app is based on Video Playback (but it's already modified a lot). I'm implementing buttons on video (which are simple textures). I'm rendering buttons (textures) on X, Y and Z axis and it works as expected.
I have a problem with detecting tap action on those buttons. Based on tap detection that is done in Video Playback I did something similar.
Video Playback uses method "projectScreenPointToPlane" and then compares "intersection" with half of the size of trackable element for each side (making 3d problem a 2d problem). However, I think that it doesn't work correctly. Video targets are quite big so it's hard to debug but I noticed the problem when I created smaller tappable areas which are buttons.
As far as I understand intersection from method "projectScreenPointToPlane" should give X and Y from the CENTER of trackable right? From my testing X and Y values change a lot depending on the camera angle and trackable position. If I move my trackable to the center of screen than values are correct.
This is easy to reproduce (debug). Just put "NSLog(@"x: %f y: %f", intersection.data[0], intersection.data[1]);" in method "tapInsideTargetWithID" in file EAGLView.mm. Open the app, move the camera to different angles and try to tap on a center of the trackable. As far as I understand it should give you values near to 0 (if you tap in the center) but it gives different values based on camera position.
I'm I doing something wrong and "projectScreenPointToPlane" works in a different way?
// Determine whether a screen tap is inside the target - (int)tapInsideTargetWithID { QCAR::Vec3F intersection, lineStart, lineEnd; // Get the current projection matrix QCAR::Matrix44F projectionMatrix = [[QCARControl getInstance] projectionMatrix]; QCAR::Matrix44F inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix); CGRect rect = [self bounds]; int touchInTarget = -1; // ----- Synchronise data access ----- [dataLock lock]; // The target returns as pose the centre of the trackable. Thus its // dimensions go from -width / 2 to width / 2 and from -height / 2 to // height / 2. The following if statement simply checks that the tap is // within this range for (int i = 0; i < NUM_VIDEO_TARGETS; ++i) { SampleMath::projectScreenPointToPlane(inverseProjMatrix, videoData[i].modelViewMatrix, rect.size.width, rect.size.height, QCAR::Vec2F(touchLocation_X, touchLocation_Y), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd); NSLog(@"x: %f y: %f", intersection.data[0], intersection.data[1]); if ((intersection.data[0] >= -videoData[i].targetPositiveDimensions.data[0]) && (intersection.data[0] <= videoData[i].targetPositiveDimensions.data[0]) && (intersection.data[1] >= -videoData[i].targetPositiveDimensions.data[1]) && (intersection.data[1] <= videoData[i].targetPositiveDimensions.data[1])) { // The tap is only valid if it is inside an active target if (YES == videoData[i].isActive) { touchInTarget = i; break; } } } [dataLock unlock]; // ----- End synchronise data access ----- return touchInTarget; }
I only used this with QCARUtils (which is used across most of the samples) in version 2.0.7. Version 2.6.6. of VideoPlayback uses something different but no problem I will track the problem tomorrow and send you code for 2.6.6 :)