The isTapOnScreenInsideTarget method is supposed to return true if the tap is in an area that covers the Image Target screen area (i.e. the projected area of the target rectangle onto the screen), not on a "3D model".

in Vuforia 4.0 you can see the same code written in Java.

If you need to tap on a 3D model, that's something that can be done at OpenGL and Android programming level (there is no Vuforia API dedicated to achieving that).

Hi AlessandroB, I have been able to run it sucesssfully. But the isTapOnScreenInsideTarget function is giving me true for any tap on the screen irrespective of weather it is on model or not. Let me know how to go get tap only on 3d Model.

Hi Alessandro,

As per your advice I am referring VideoPlayback from 2.0.30 SDK in 4.0.103 SDk since there is no JNI in the latter one.

ImageTarget.cpp

#include <QCAR/ImageTarget.h>
#include <QCAR/Vectors.h>



#include "SampleMath.h"


...

// The projection matrix used for rendering virtual objects:
QCAR::Matrix44F projectionMatrix;

QCAR::Matrix44F inverseProjMatrix;

// Trackable dimensions
QCAR::Vec2F targetPositiveDimensions[1];

// Needed to calculate whether a screen tap is inside the target
QCAR::Matrix44F modelViewMatrix;

void projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal,
                               QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd);
bool linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd, QCAR::Vec3F pointOnPlane,
                           QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection);


...

JNIEXPORT bool JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_isTapOnScreenInsideTarget(JNIEnv *env, jobject, jint target, jfloat x, jfloat y)
{
    //LOG("Java_com_qualcomm_QCARSamples_VideoPlayback_VideoPlayback_isTapOnScreenInsideTarget");
    // Here we calculate that the touch event is inside the target
    QCAR::Vec3F intersection, lineStart, lineEnd;

    projectScreenPointToPlane(QCAR::Vec2F(x, y), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);

    // The target returns as pose the center of the trackable. The following if-statement simply checks that the tap is within this range
    if ( (intersection.data[0] >= -(targetPositiveDimensions[target].data[0])) && (intersection.data[0] <= (targetPositiveDimensions[target].data[0])) &&
         (intersection.data[1] >= -(targetPositiveDimensions[target].data[1])) && (intersection.data[1] <= (targetPositiveDimensions[target].data[1])))
        return true;
    else
        return false;
}

...

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_setProjectionMatrix(JNIEnv *, jobject)
{
    LOG("Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_setProjectionMatrix");

    // Cache the projection matrix:
    const QCAR::CameraCalibration& cameraCalibration =
                                QCAR::CameraDevice::getInstance().getCameraCalibration();
    projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 10.0f, 5000.0f);

    //Added
    inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);

}

...


JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject)
{

...

    targetPositiveDimensions[0].data[0] = 0.0;
    targetPositiveDimensions[0].data[1] = 0.0;

    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++)
    {
        // Get the trackable:
        const QCAR::TrackableResult* result = state.getTrackableResult(tIdx);
        const QCAR::Trackable& trackable = result->getTrackable();
        const QCAR::ImageTarget* imageTarget = (const QCAR::ImageTarget) result->getTrackable();
        //QCAR::Vec2F targetSize = ((QCAR::ImageTarget *) trackable)->getSize();
        //QCAR::Vec2F targetSize = trackable->getSize();

        modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(result->getPose());

        targetPositiveDimensions[0] = imagetarget->getSize();

        // The pose delivers the center of the target, thus the dimensions
        // go from -width/2 to width/2, same for height
        targetPositiveDimensions[0].data[0] /= 2.0f;
        targetPositiveDimensions[0].data[1] /= 2.0f;


        if(!isExtendedTrackingActivated)
        {
            // Choose the texture based on the target name:
            int textureIndex;
            if (strcmp(trackable.getName(), "chips") == 0)
            {...

}


// ----------------------------------------------------------------------------
// From Dominoes.cpp
// ----------------------------------------------------------------------------

void
projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal,
                          QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd)
{
...
}


bool
linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd,
                      QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal,
                      QCAR::Vec3F &intersection)
{

...
}

ImageTarget.java


    /*
     *  Native method to check if the tap is made inside the target
     */
    private native boolean isTapOnScreenInsideTarget(int target, float x, float y);

@Override
        public boolean onDoubleTap(MotionEvent e) {
            // Verify that the tap happens inside the target:
            if (isTapOnScreenInsideTarget(0, e.getX(), e.getY()))
            {
                //TODO: Target is Double Tapped
            }
            return true;
        }

OUTPUT

[armeabi-v7a] Compile++ arm  : ImageTargetsNative <= ImageTargets.cpp
jni/ImageTargets.cpp: In function 'void Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv*, jobject)':
jni/ImageTargets.cpp:392:39: error: 'imagetarget' was not declared in this scope
         targetPositiveDimensions[0] = imagetarget->getSize();
                                       ^
jni/ImageTargets.cpp: In function 'bool linePlaneIntersection(QCAR::Vec3F, QCAR::Vec3F, QCAR::Vec3F, QCAR::Vec3F, QCAR::Vec3F&)':
jni/ImageTargets.cpp:912:15: error: 'fabs' was not declared in this scope
     if (fabs(d) < 0.00001) {
               ^
make: *** [obj/local/armeabi-v7a/objs/ImageTargetsNative/ImageTargets.o] Error 1

Please guide me how to get rid of the errors.

There is a good example of that in the VideoPlayback sample (see VideoPlayback.java and VideoPlayback.cpp in particular).

Can anyone give me a hint on how to modify the same code in order to get a tap event for tapping the texture overlaid on the trackable image.

You just need to fix C++ compile errors,

for instance in this line:

dear Al, please help me with this problem

thanks,

regards

hi All,

I have an issue when implement the code bellow (Kim SUggestion)

and here the log :

C:\Development\Android\android-vuforia-sdk-2-0-31\samples\ImageTargets-2-0-7>ndk
-build
"Compile++ arm  : ImageTargets <= ImageTargets.cpp
jni/ImageTargets.cpp: In function 'void Java_com_qualcomm_QCARSamples_ImageTarge
ts_ImageTargetsRenderer_renderFrame(JNIEnv*, jobject)':
jni/ImageTargets.cpp:378:22: error: base operand of '->' has non-pointer type 'c
onst QCAR::Trackable'
jni/ImageTargets.cpp:379:26: error: 'env' was not declared in this scope
jni/ImageTargets.cpp:380:52: error: 'obj' was not declared in this scope
jni/ImageTargets.cpp:430:38: error: 'thisTexture' was not declared in this scope

jni/ImageTargets.cpp: At global scope:
jni/ImageTargets.cpp:442:14: error: expected constructor, destructor, or type co
nversion before '(' token
jni/ImageTargets.cpp:450:31: error: expected constructor, destructor, or type co
nversion before '(' token
jni/ImageTargets.cpp:451:31: error: expected constructor, destructor, or type co
nversion before '(' token
jni/ImageTargets.cpp:452:31: error: expected constructor, destructor, or type co
nversion before '(' token
jni/ImageTargets.cpp:455:34: error: expected constructor, destructor, or type co
nversion before '.' token
jni/ImageTargets.cpp:766:1: error: expected declaration before '}' token
make: *** [obj/local/armeabi/objs/ImageTargets/ImageTargets.o] Error 1

could somebody tell me what is my wrong

Edit: I solved my problem, Thanks guys.

hi Kim, 

I follwed your indications , but I am getting the NaN values , could you help please.

I cached both matrics, here is some snippets from my code :

Hi giagi87,

tapping precisely on the 3D model would require a sophisticated raycasting algorithm testing the intersection on every triangle of your model;

this is technically possible, but the implemention of such approach is far beyond the intent and scope of the checkIntersectionLine method that you refer to.

For those advanced use cases you may want to take into consideration the integration of 3rd party game engines (or 3D rendering engines) which support such high level functionalities (if you search this forum you may find that some people have successfully integrated engines like Ogre, JPCT, Adobe AIR, to name a few...)

Another possible option would be to use Unity with the Vuforia Unity extension, and build for the Android target platform.

I have the same problem.

the code posted by yaya return intersectionCheck=true  anywhere i tap in the screen.

instead i want that checkIntersectionLine(modelViewProjection, lineStart, lineEnd); function return true only when i tap on 3d model,

not when i tap on trackable.

had someone resolve this question?

thanks in advance.

giacomo

@charsanova:

pls refer to https://ar.qualcomm.at/node/2001229

const QCAR::CameraCalibration& cameraCalibration = QCAR::CameraDevice::getInstance().getCameraCalibration();

Hey,

Thanks for you help so far, I appreciate it so much! Kim you are a lifesaver! 

I too am trying to add a tap event onto my object. I have copied the snippets below into (hopefully) the relevant places, however now when I compile I am getting a whole load of messy errors, I have taken out all of the <p>, </br>, ... and </p> etc as these seemed to be making problems and I might be wrong but didnt feel they were necessary. After this, the first error that I am getting is that 'state' is not declared in this scope. I can work through this myself and try to debug it all but I'm getting the feeling that something isn't right as I should just be able to drop this code in, right? Do you have any idea why it wouldnt be working for me?

Thanks again! :-) 

Help me :((

this my error :

$ /cygdrive/c/development/android/android-ndk-r7c/ndk-build
Gdbserver : [arm-linux-androideabi-4.4.3] libs/armeabi/gdbserver
Gdbsetup : libs/armeabi/gdb.setup
Gdbserver : [arm-linux-androideabi-4.4.3] libs/armeabi-v7a/gdbserver
Gdbsetup : libs/armeabi-v7a/gdb.setup
Compile++ arm : ImageTargets C:/development/android/vuforia-sdk-android-1-5-9/samples/imagetargets/jni/ImageTargets.cpp: In function 'void Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv*, _jobject*)':
C:/development/android/vuforia-sdk-android-1-5-9/samples/imagetargets/jni/ImageTargets.cpp:700: error: 'getInstance' is not a member of 'QCAR::Tracker'
make: *** [/cygdrive/c/development/android/vuforia-sdk-android-1-5-9/samples/imagetargets/obj/local/armeabi/objs-debug/ImageTargets/ImageTargets.o] Error 1

and this is my code, am i wrong?

Quote:

........................

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *,
jobject)
{
LOG("Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera");

// Initialize the camera:
if (!QCAR::CameraDevice::getInstance().init())
return;

// Configure the video background
configureVideoBackground();

// Select the default mode:
if (!QCAR::CameraDevice::getInstance().selectVideoMode(
QCAR::CameraDevice::MODE_DEFAULT))
return;

// Start the camera:
if (!QCAR::CameraDevice::getInstance().start())
return;

// Uncomment to enable flash
//if(QCAR::CameraDevice::getInstance().setFlashTorchMode(true))
// LOG("IMAGE TARGETS : enabled torch");

// Uncomment to enable infinity focus mode, or any other supported focus mode
// See CameraDevice.h for supported focus modes
//if(QCAR::CameraDevice::getInstance().setFocusMode(QCAR::CameraDevice::FOCUS_MODE_INFINITY))
// LOG("IMAGE TARGETS : enabled infinity focus");

// Start the tracker:
QCAR::TrackerManager& trackerManager = QCAR::TrackerManager::getInstance();
QCAR::Tracker* imageTracker = trackerManager.getTracker(QCAR::Tracker::IMAGE_TRACKER);
if(imageTracker != 0)
imageTracker->start();

// Cache the projection matrix:
const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
const QCAR::CameraCalibration& cameraCalibration = QCAR::CameraDevice::getInstance().getCameraCalibration();
projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f,
2000.0f);

// Cache the inverse projection matrix:
inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
}

........................

Thanks kim.. but i have got some prob..

I am also trying to implement tap events. For me it would be enough for now to get the tap coordinates in target space and pass them back to java. I implemented the suggested code changes. I still get the following error when building with ndk:

jni/ImageTargets.cpp:234: error: 'projectScreenPointToPlane' was not declared in this scope

I am rather new to jni development. Do I have to register the projectScreenPointToPlane method somewhere or what is wrong?

ksiva wrote:

The problem is that the x,y coordinates returned by a tap are in screen space. You'll want to project that to target space to determine whether or not a target was touched.

Here are the steps for determining whether a target was touched in the ImageTargets sample.

1. Copy SampleMath.h and SampleMath.cpp from Dominoes/jni to ImageTargets/jni.
2. Add SampleMath.cpp to the LOCAL_SRC_FILES variable in ImageTargets/jni/Android.mk.

3. Make the following changes to ImageTargets.cpp:

#include "SampleMath.h"
#include <QCAR/ImageTarget.h>
#include <math.h>

QCAR::Matrix44F inverseProjMatrix;
QCAR::Matrix44F modelViewMatrix;

bool tap = false;
float tapX = 0;
float tapY = 0;

JNIEXPORT int JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_nativeTapEvent(JNIEnv *, jobject, jfloat x, jfloat y)
{
    tapX = x;
    tapY = y;
    tap = true;
}

// Copy this from Dominoes.cpp
bool
linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd,
                      QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal,
                      QCAR::Vec3F &intersection)
{
    ...
}

// Copy this from Dominoes.cpp
void
projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal,
                          QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd)
{
    ...
}

// Edit existing method
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject)
{
    ...
    
    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
    {
        // Get the trackable:
        const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
        //QCAR::Matrix44F modelViewMatrix =
        //    QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
        
        // Cache the modelViewMatrix
        modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
        
        if (tap)
        {
            QCAR::Vec3F intersection, lineStart, lineEnd;
            projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);
            
            const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable);
            QCAR::Vec2F trackableSize = imageTarget->getSize();
            
            LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY);
            LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]);
            
            if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) &&
                fabs(intersection.data[1]) < (trackableSize.data[1] / 2))
            {
                LOG("tapped inside the target!");
                // do something here
            }
            
            tap = false;
        }
        
        ...
    }
    ...
}

// Edit existing method
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *, jobject)
{
    ...
    
    // Cache the projection matrix:
    const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
    const QCAR::CameraCalibration& cameraCalibration =
                                    tracker.getCameraCalibration();
    projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f,
                                            2000.0f);
    
    // Cache the inverse projection matrix:
    inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
}

4. Make the following changes to ImageTargets.java:

    private GestureDetector mGestureDetector;
    
    // Edit existing method
    protected void onCreate(Bundle savedInstanceState)
    {
        ...
        
        mGestureDetector = new GestureDetector(this, new TapGestureListener());
    }
    
    @Override
    public boolean onTouchEvent(MotionEvent event)
    {
        if (mGestureDetector.onTouchEvent(event))
            return true;
        else
            return false;
    }
    
    private native boolean nativeTapEvent(float tapX, float tapY);
    
    class TapGestureListener extends GestureDetector.SimpleOnGestureListener
    {
        @Override
        public boolean onSingleTapUp(MotionEvent ev)
        {
            nativeTapEvent(ev.getX(), ev.getY());
            return true;
        }
    }

- Kim

You need to make sure that you are using the modelview matrix for the correct target when using the code from the Dominoes sample. You may want to adapt the intersection methods to take the modelview matrix as an argument, rather than querying it globally.

- Kim

I could open a new topic but the issue is related with this tap method. Forget about 1st issue. I need tap event for multiple targets. How can I accomplish this by using this method. I implemented this method for one target succesfully; however, it does not work properly when it encountered by two or more target.

It isn't clear what you are trying to do. Please open a new thread and explain what you want to accomplish and what isn't working. Thanks,

- Kim

Does anyone has information about that?

I succesfully implemented tap event; however, there are two issue;

1 - When camera position is moved to up or down, as natural the target view is scaled in the real world and this event can detect that, before moving its height is for example, 400 unit, after moving it became 250 unit and this tap event can detect the change and the event is not invoked according to previous target position. But the problem is, there is no same issue about moving to right or left. When we move the camera to right or left, its real world view to the camera is changed but the event is invoked according to default width of the target. When we tap out of the target, it is logging inside the target.

2 - What about with multiple targets? When we try that implementation with two target, it is not processed. It is invoked when we tap somewhere else from targets.

studing........

QCAR 1.5 changed a few API calls. Try this:

const QCAR::CameraCalibration& cameraCalibration = QCAR::CameraDevice::getInstance().getCameraCalibration();

- Kim

i tried the same but during cygwin build getting error in ImageTarget.cpp---
1.getInstance is not member of qcar::Tracker.
2.qcar::Tracker has no member named getCameraCalibration.
can anyone help me with this?

The checkIntersectionLine function only works with cubes, it cannot be used to check for intersections against a mesh. You'll need to create a bounding box around your mesh to use that function. If you want finer-grained selection you'll have to find another solution, sorry!

- Kim

Hi Kim
Thanks for your help on marker tapping. Now, we are trying the object tapping on teapot. But getting some compilation errors in the following function.

We copied the code from Dominos for doing the object tapping instead of marker tapping.


bool
checkIntersectionLine(QCAR::Matrix44F transformA, QCAR::Vec3F pointA, QCAR::Vec3F pointB)
{
...
...

QCAR::Vec3F normalA1 = SampleMath::Vec3FTransformNormal(teapotNormals[0], transformA);
...
....
}

Q1. We are getting the following error

error: no matching function for call to 'SampleMath::Vec3FTransformNormal(float&, QCAR::Matrix44F&)'
: candidates are: static QCAR::Vec3F SampleMath::Vec3FTransformNormal(QCAR::Vec3F&, QCAR::Matrix44F&)

We removed the static keyword that was used in the declaration of teapotNormals

Q2. In the case of dominos it was a cube, we had only 9 values for dominosNOrmals. So we called the
function SampleMath::Vec3FTransformNormal 3 times.

But if we use some other object like a teapot, the number of values in the teapotNOrmals is NUM_TEAPOT_OBJECT_VERTEX * 3 ie (824*3). So
can you please let us know how to pass the n number of values of teapotNOrmals to the function SampleMath::Vec3FTransformNormal
Is it like we have to call function SampleMath::Vec3FTransformNormal 824 times in a loop?

Regards
gilesian

Sorry, that's outside the scope of what I can help with. You'll want to research 3D object picking, there are lots of approaches to this problem.

The Dominoes sample uses one approach, with object-aligned bounding boxes that surround the 3D models (you have to define these). See the checkIntersectionLine() method in Dominoes.cpp. "dominoNormals" and "dominoBaseVertices" just define a unit cube, and the "transformA" matrix is used to scale, rotate, and translate that cube to form a box that surrounds the model. Then it performs an intersection test with the line defined by "pointA" and "pointB". Trace through the code and see how that function is used, it might provide a good starting point.

- Kim

that would interest me too!

Now, how can a get a tap event on the object itself. What changes needs to be done in the above code mentioned by you.

Thanks

Hi Kim,

Thanks a lot for all your help and support.

We are successful in getting a Tap event on Markers.

Thanks

Yes, sorry, tapX and tapY should be floats (I've fixed the original post).

You would get NaN values if you forgot to set the global inverseProjMatrix or modelViewMatrix. Make sure you cache both of these.

        // Cache the modelViewMatrix
        modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(trackable->getPose());

- Kim

Hi Kim,

Thanks for the help..i implemented all the changes as u mentioned and we are able to get tap event but, we are unable to get the LinePlaneIntersection values, we are getting as NaN.

please let us know what exactly we are missing ...Thanks in advance

we change the data type of tapX and tapY variables to float in ImageTarget.cpp file..

Thanks

*Native tap coordinates (*Native screen space): 420.00, 168.00
***the x value420.0 (15717): the y value168.0
setPreviewCallbackFlag (pid 15717)
*point values from screentoplane**: 420.00, 168.00
**point values from screentoplane**: 0.05, 0.24
LinePLane Inetersection values*: LinePLane Inetersection values*: %.2f, %.2f, NaN,NaN
tap coordinates (screen space): 420.00, 168.00
tap coordinates (target space): NaN, NaN

ksiva wrote:

The problem is that the x,y coordinates returned by a tap are in screen space. You'll want to project that to target space to determine whether or not a target was touched.

Here are the steps for determining whether a target was touched in the ImageTargets sample.

1. Copy SampleMath.h and SampleMath.cpp from Dominoes/jni to ImageTargets/jni.
2. Add SampleMath.cpp to the LOCAL_SRC_FILES variable in ImageTargets/jni/Android.mk.

3. Make the following changes to ImageTargets.cpp:

#include "SampleMath.h"
#include <QCAR/ImageTarget.h>
#include <math.h>

QCAR::Matrix44F inverseProjMatrix;
QCAR::Matrix44F modelViewMatrix;

bool tap = false;
int tapX = 0;
int tapY = 0;

JNIEXPORT int JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_nativeTapEvent(JNIEnv *, jobject, jfloat x, jfloat y)
{
    tapX = x;
    tapY = y;
    tap = true;
}

// Copy this from Dominoes.cpp
bool
linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd,
                      QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal,
                      QCAR::Vec3F &intersection)
{
    ...
}

// Copy this from Dominoes.cpp
void
projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal,
                          QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd)
{
    ...
}

// Edit existing method
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject)
{
    ...
    
    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
    {
        // Get the trackable:
        const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
        //QCAR::Matrix44F modelViewMatrix =
        //    QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
        
        // Cache the modelViewMatrix
        modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
        
        if (tap)
        {
            QCAR::Vec3F intersection, lineStart, lineEnd;
            projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);
            
            const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable);
            QCAR::Vec2F trackableSize = imageTarget->getSize();
            
            LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY);
            LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]);
            
            if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) &&
                fabs(intersection.data[1]) < (trackableSize.data[1] / 2))
            {
                LOG("tapped inside the target!");
                // do something here
            }
            
            tap = false;
        }
        
        ...
    }
    ...
}

// Edit existing method
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *, jobject)
{
    ...
    
    // Cache the projection matrix:
    const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
    const QCAR::CameraCalibration& cameraCalibration =
                                    tracker.getCameraCalibration();
    projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f,
                                            2000.0f);
    
    // Cache the inverse projection matrix:
    inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
}

4. Make the following changes to ImageTargets.java:

    private GestureDetector mGestureDetector;
    
    // Edit existing method
    protected void onCreate(Bundle savedInstanceState)
    {
        ...
        
        mGestureDetector = new GestureDetector(this, new TapGestureListener());
    }
    
    @Override
    public boolean onTouchEvent(MotionEvent event)
    {
        if (mGestureDetector.onTouchEvent(event))
            return true;
        else
            return false;
    }
    
    private native boolean nativeTapEvent(float tapX, float tapY);
    
    class TapGestureListener extends GestureDetector.SimpleOnGestureListener
    {
        @Override
        public boolean onSingleTapUp(MotionEvent ev)
        {
            nativeTapEvent(ev.getX(), ev.getY());
            return true;
        }
    }

- Kim

The problem is that the x,y coordinates returned by a tap are in screen space. You'll want to project that to target space to determine whether or not a target was touched.

Here are the steps for determining whether a target was touched in the ImageTargets sample.

1. Copy SampleMath.h and SampleMath.cpp from Dominoes/jni to ImageTargets/jni.
2. Add SampleMath.cpp to the LOCAL_SRC_FILES variable in ImageTargets/jni/Android.mk.

3. Make the following changes to ImageTargets.cpp:

#include "SampleMath.h"
#include <QCAR/ImageTarget.h>
#include <math.h>

QCAR::Matrix44F inverseProjMatrix;
QCAR::Matrix44F modelViewMatrix;

bool tap = false;
float tapX = 0;
float tapY = 0;

JNIEXPORT int JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_nativeTapEvent(JNIEnv *, jobject, jfloat x, jfloat y)
{
    tapX = x;
    tapY = y;
    tap = true;
}

// Copy this from Dominoes.cpp
bool
linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd,
                      QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal,
                      QCAR::Vec3F &intersection)
{
    ...
}

// Copy this from Dominoes.cpp
void
projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal,
                          QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd)
{
    ...
}

// Edit existing method
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject)
{
    ...
    
    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
    {
        // Get the trackable:
        const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
        //QCAR::Matrix44F modelViewMatrix =
        //    QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
        
        // Cache the modelViewMatrix
        modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
        
        if (tap)
        {
            QCAR::Vec3F intersection, lineStart, lineEnd;
            projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);
            
            const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable);
            QCAR::Vec2F trackableSize = imageTarget->getSize();
            
            LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY);
            LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]);
            
            if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) &&
                fabs(intersection.data[1]) < (trackableSize.data[1] / 2))
            {
                LOG("tapped inside the target!");
                // do something here
            }
            
            tap = false;
        }
        
        ...
    }
    ...
}

// Edit existing method
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *, jobject)
{
    ...
    
    // Cache the projection matrix:
    const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
    const QCAR::CameraCalibration& cameraCalibration =
                                    tracker.getCameraCalibration();
    projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f,
                                            2000.0f);
    
    // Cache the inverse projection matrix:
    inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
}

4. Make the following changes to ImageTargets.java:

    private GestureDetector mGestureDetector;
    
    // Edit existing method
    protected void onCreate(Bundle savedInstanceState)
    {
        ...
        
        mGestureDetector = new GestureDetector(this, new TapGestureListener());
    }
    
    @Override
    public boolean onTouchEvent(MotionEvent event)
    {
        if (mGestureDetector.onTouchEvent(event))
            return true;
        else
            return false;
    }
    
    private native boolean nativeTapEvent(float tapX, float tapY);
    
    class TapGestureListener extends GestureDetector.SimpleOnGestureListener
    {
        @Override
        public boolean onSingleTapUp(MotionEvent ev)
        {
            nativeTapEvent(ev.getX(), ev.getY());
            return true;
        }
    }

- Kim

Hi kim ... so now what I'm planing to do is:

1. get x,y (in java) of the tap and pass it to native code
2. get the area of marker.
3. check if x,y lies in markers area.

Please tell me how can i achieve above thing .. also provide some code snippet ..

also i did not understand your 2nd suggestion "Or, you could use the projectScreenPointToPlane method from that sample to get the touch coordinates on the 2D marker plane." .. can u plz elaborate

hi kim ... im droping this ... because i feel its 2 difficult .. instead what ill try to find out is "Tap on marker"(on device screen). please provide me with some code snippet for the same ... Note: i have tried virtual button ... the problem with that is for virtual button we have to tap on the marker present on the paper and not on the device screen...

thanks kim .. i have started looking into dominoes example ... can u plzx tell me how can i debug cpp code or how can i but debug statements ???

Handling tap events isn't built into the SDK. The Dominoes sample does have some code for selecting 3D objects on tap, but you'll have to dig into it a bit to reuse it. That sample intersects the touch against object-orientented bounding boxes (you have to define these per 3D object). Or, you could use the projectScreenPointToPlane method from that sample to get the touch coordinates on the 2D marker plane.

- Kim

how can i also get a tap event for the marker. i am very new to qcar, so can u please tell me where can i find the concerned code and also what code change needs to be done

Hi,
i am also trying the same but i am not able to get the results.
i m confused ..

please let me know the if u got the results and the correct approach.

Thanks
Atik