Hi, I am working on the ImageTargets project. What i want to do is On Tap of 3D Model/Object i want to start a new new Activity in Android. So, how can i get a onClick event on the 3D object. Thanks.
- Sort Posts
- 41 replies
- Last post
How to get a tap event on the 3D model.
Hi AlessandroB, I have been able to run it sucesssfully. But the isTapOnScreenInsideTarget function is giving me true for any tap on the screen irrespective of weather it is on model or not. Let me know how to go get tap only on 3d Model.
How to get a tap event on the 3D model.
Hi Alessandro,
As per your advice I am referring VideoPlayback from 2.0.30 SDK in 4.0.103 SDk since there is no JNI in the latter one.
ImageTarget.cpp #include <QCAR/ImageTarget.h> #include <QCAR/Vectors.h> #include "SampleMath.h" ... // The projection matrix used for rendering virtual objects: QCAR::Matrix44F projectionMatrix; QCAR::Matrix44F inverseProjMatrix; // Trackable dimensions QCAR::Vec2F targetPositiveDimensions[1]; // Needed to calculate whether a screen tap is inside the target QCAR::Matrix44F modelViewMatrix; void projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd); bool linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd, QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection); ... JNIEXPORT bool JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_isTapOnScreenInsideTarget(JNIEnv *env, jobject, jint target, jfloat x, jfloat y) { //LOG("Java_com_qualcomm_QCARSamples_VideoPlayback_VideoPlayback_isTapOnScreenInsideTarget"); // Here we calculate that the touch event is inside the target QCAR::Vec3F intersection, lineStart, lineEnd; projectScreenPointToPlane(QCAR::Vec2F(x, y), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd); // The target returns as pose the center of the trackable. The following if-statement simply checks that the tap is within this range if ( (intersection.data[0] >= -(targetPositiveDimensions[target].data[0])) && (intersection.data[0] <= (targetPositiveDimensions[target].data[0])) && (intersection.data[1] >= -(targetPositiveDimensions[target].data[1])) && (intersection.data[1] <= (targetPositiveDimensions[target].data[1]))) return true; else return false; } ... JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_setProjectionMatrix(JNIEnv *, jobject) { LOG("Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_setProjectionMatrix"); // Cache the projection matrix: const QCAR::CameraCalibration& cameraCalibration = QCAR::CameraDevice::getInstance().getCameraCalibration(); projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 10.0f, 5000.0f); //Added inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix); } ... JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject) { ... targetPositiveDimensions[0].data[0] = 0.0; targetPositiveDimensions[0].data[1] = 0.0; // Did we find any trackables this frame? for(int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++) { // Get the trackable: const QCAR::TrackableResult* result = state.getTrackableResult(tIdx); const QCAR::Trackable& trackable = result->getTrackable(); const QCAR::ImageTarget* imageTarget = (const QCAR::ImageTarget) result->getTrackable(); //QCAR::Vec2F targetSize = ((QCAR::ImageTarget *) trackable)->getSize(); //QCAR::Vec2F targetSize = trackable->getSize(); modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(result->getPose()); targetPositiveDimensions[0] = imagetarget->getSize(); // The pose delivers the center of the target, thus the dimensions // go from -width/2 to width/2, same for height targetPositiveDimensions[0].data[0] /= 2.0f; targetPositiveDimensions[0].data[1] /= 2.0f; if(!isExtendedTrackingActivated) { // Choose the texture based on the target name: int textureIndex; if (strcmp(trackable.getName(), "chips") == 0) {... } // ---------------------------------------------------------------------------- // From Dominoes.cpp // ---------------------------------------------------------------------------- void projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd) { ... } bool linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd, QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection) { ... }
ImageTarget.java /* * Native method to check if the tap is made inside the target */ private native boolean isTapOnScreenInsideTarget(int target, float x, float y); @Override public boolean onDoubleTap(MotionEvent e) { // Verify that the tap happens inside the target: if (isTapOnScreenInsideTarget(0, e.getX(), e.getY())) { //TODO: Target is Double Tapped } return true; }
OUTPUT
[armeabi-v7a] Compile++ arm : ImageTargetsNative <= ImageTargets.cpp jni/ImageTargets.cpp: In function 'void Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv*, jobject)': jni/ImageTargets.cpp:392:39: error: 'imagetarget' was not declared in this scope targetPositiveDimensions[0] = imagetarget->getSize(); ^ jni/ImageTargets.cpp: In function 'bool linePlaneIntersection(QCAR::Vec3F, QCAR::Vec3F, QCAR::Vec3F, QCAR::Vec3F, QCAR::Vec3F&)': jni/ImageTargets.cpp:912:15: error: 'fabs' was not declared in this scope if (fabs(d) < 0.00001) { ^ make: *** [obj/local/armeabi-v7a/objs/ImageTargetsNative/ImageTargets.o] Error 1
Please guide me how to get rid of the errors.
How to get a tap event on the 3D model.
There is a good example of that in the VideoPlayback sample (see VideoPlayback.java and VideoPlayback.cpp in particular).
How to get a tap event on the 3D model.
Can anyone give me a hint on how to modify the same code in order to get a tap event for tapping the texture overlaid on the trackable image.
How to get a tap event on the 3D model.
You just need to fix C++ compile errors,
for instance in this line:
void
Java_com_qualcomm_QCARSamples_ImageTarge
ts_ImageTargetsRenderer_renderFrame(JNIEnv*, jobject)':void
Java_com_qualcomm_QCARSamples_ImageTarge
ts_ImageTargetsRenderer_renderFrame(JNIEnv* env, jobject obj)'
How to get a tap event on the 3D model.
hi All,
I have an issue when implement the code bellow (Kim SUggestion)
and here the log :
C:\Development\Android\android-vuforia-sdk-2-0-31\samples\ImageTargets-2-0-7>ndk -build "Compile++ arm : ImageTargets <= ImageTargets.cpp jni/ImageTargets.cpp: In function 'void Java_com_qualcomm_QCARSamples_ImageTarge ts_ImageTargetsRenderer_renderFrame(JNIEnv*, jobject)': jni/ImageTargets.cpp:378:22: error: base operand of '->' has non-pointer type 'c onst QCAR::Trackable' jni/ImageTargets.cpp:379:26: error: 'env' was not declared in this scope jni/ImageTargets.cpp:380:52: error: 'obj' was not declared in this scope jni/ImageTargets.cpp:430:38: error: 'thisTexture' was not declared in this scope jni/ImageTargets.cpp: At global scope: jni/ImageTargets.cpp:442:14: error: expected constructor, destructor, or type co nversion before '(' token jni/ImageTargets.cpp:450:31: error: expected constructor, destructor, or type co nversion before '(' token jni/ImageTargets.cpp:451:31: error: expected constructor, destructor, or type co nversion before '(' token jni/ImageTargets.cpp:452:31: error: expected constructor, destructor, or type co nversion before '(' token jni/ImageTargets.cpp:455:34: error: expected constructor, destructor, or type co nversion before '.' token jni/ImageTargets.cpp:766:1: error: expected declaration before '}' token make: *** [obj/local/armeabi/objs/ImageTargets/ImageTargets.o] Error 1
could somebody tell me what is my wrong
How to get a tap event on the 3D model.
hi Kim,
I follwed your indications , but I am getting the NaN values , could you help please.
I cached both matrics, here is some snippets from my code :
How to get a tap event on the 3D model.
Hi giagi87,
tapping precisely on the 3D model would require a sophisticated raycasting algorithm testing the intersection on every triangle of your model;
this is technically possible, but the implemention of such approach is far beyond the intent and scope of the checkIntersectionLine method that you refer to.
For those advanced use cases you may want to take into consideration the integration of 3rd party game engines (or 3D rendering engines) which support such high level functionalities (if you search this forum you may find that some people have successfully integrated engines like Ogre, JPCT, Adobe AIR, to name a few...)
Another possible option would be to use Unity with the Vuforia Unity extension, and build for the Android target platform.
How to get a tap event on the 3D model.
I have the same problem.
the code posted by yaya return intersectionCheck=true anywhere i tap in the screen.
instead i want that checkIntersectionLine(modelViewProjection, lineStart, lineEnd); function return true only when i tap on 3d model,
not when i tap on trackable.
had someone resolve this question?
thanks in advance.
giacomo
How to get a tap event on the 3D model.
Hey,
Thanks for you help so far, I appreciate it so much! Kim you are a lifesaver!
I too am trying to add a tap event onto my object. I have copied the snippets below into (hopefully) the relevant places, however now when I compile I am getting a whole load of messy errors, I have taken out all of the <p>, </br>, ... and </p> etc as these seemed to be making problems and I might be wrong but didnt feel they were necessary. After this, the first error that I am getting is that 'state' is not declared in this scope. I can work through this myself and try to debug it all but I'm getting the feeling that something isn't right as I should just be able to drop this code in, right? Do you have any idea why it wouldnt be working for me?
Thanks again! :-)
Re: How to get a tap event on the 3D model.
Help me :((
this my error :
$ /cygdrive/c/development/android/android-ndk-r7c/ndk-build
Gdbserver : [arm-linux-androideabi-4.4.3] libs/armeabi/gdbserver
Gdbsetup : libs/armeabi/gdb.setup
Gdbserver : [arm-linux-androideabi-4.4.3] libs/armeabi-v7a/gdbserver
Gdbsetup : libs/armeabi-v7a/gdb.setup
Compile++ arm : ImageTargets
C:/development/android/vuforia-sdk-android-1-5-9/samples/imagetargets/jni/ImageTargets.cpp: In function 'void Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv*, _jobject*)':
C:/development/android/vuforia-sdk-android-1-5-9/samples/imagetargets/jni/ImageTargets.cpp:700: error: 'getInstance' is not a member of 'QCAR::Tracker'
make: *** [/cygdrive/c/development/android/vuforia-sdk-android-1-5-9/samples/imagetargets/obj/local/armeabi/objs-debug/ImageTargets/ImageTargets.o] Error 1
and this is my code, am i wrong?
........................
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *,
jobject)
{
LOG("Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera");// Initialize the camera:
if (!QCAR::CameraDevice::getInstance().init())
return;// Configure the video background
configureVideoBackground();// Select the default mode:
if (!QCAR::CameraDevice::getInstance().selectVideoMode(
QCAR::CameraDevice::MODE_DEFAULT))
return;// Start the camera:
if (!QCAR::CameraDevice::getInstance().start())
return;// Uncomment to enable flash
//if(QCAR::CameraDevice::getInstance().setFlashTorchMode(true))
// LOG("IMAGE TARGETS : enabled torch");// Uncomment to enable infinity focus mode, or any other supported focus mode
// See CameraDevice.h for supported focus modes
//if(QCAR::CameraDevice::getInstance().setFocusMode(QCAR::CameraDevice::FOCUS_MODE_INFINITY))
// LOG("IMAGE TARGETS : enabled infinity focus");// Start the tracker:
QCAR::TrackerManager& trackerManager = QCAR::TrackerManager::getInstance();
QCAR::Tracker* imageTracker = trackerManager.getTracker(QCAR::Tracker::IMAGE_TRACKER);
if(imageTracker != 0)
imageTracker->start();// Cache the projection matrix:
const QCAR::Tracker& tracker = QCAR::Tracker::getInstance();
const QCAR::CameraCalibration& cameraCalibration = QCAR::CameraDevice::getInstance().getCameraCalibration();
projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f,
2000.0f);// Cache the inverse projection matrix:
inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix);
}........................
Re: How to get a tap event on the 3D model.
Thanks kim.. but i have got some prob..
Re: How to get a tap event on the 3D model.
I am also trying to implement tap events. For me it would be enough for now to get the tap coordinates in target space and pass them back to java. I implemented the suggested code changes. I still get the following error when building with ndk:
jni/ImageTargets.cpp:234: error: 'projectScreenPointToPlane' was not declared in this scope
I am rather new to jni development. Do I have to register the projectScreenPointToPlane method somewhere or what is wrong?
The problem is that the x,y coordinates returned by a tap are in screen space. You'll want to project that to target space to determine whether or not a target was touched.
Here are the steps for determining whether a target was touched in the ImageTargets sample.
1. Copy SampleMath.h and SampleMath.cpp from Dominoes/jni to ImageTargets/jni.
2. Add SampleMath.cpp to the LOCAL_SRC_FILES variable in ImageTargets/jni/Android.mk.3. Make the following changes to ImageTargets.cpp:
#include "SampleMath.h" #include <QCAR/ImageTarget.h> #include <math.h> QCAR::Matrix44F inverseProjMatrix; QCAR::Matrix44F modelViewMatrix; bool tap = false; float tapX = 0; float tapY = 0; JNIEXPORT int JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_nativeTapEvent(JNIEnv *, jobject, jfloat x, jfloat y) { tapX = x; tapY = y; tap = true; } // Copy this from Dominoes.cpp bool linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd, QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection) { ... } // Copy this from Dominoes.cpp void projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd) { ... } // Edit existing method JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject) { ... // Did we find any trackables this frame? for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++) { // Get the trackable: const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx); //QCAR::Matrix44F modelViewMatrix = // QCAR::Tool::convertPose2GLMatrix(trackable->getPose()); // Cache the modelViewMatrix modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose()); if (tap) { QCAR::Vec3F intersection, lineStart, lineEnd; projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd); const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable); QCAR::Vec2F trackableSize = imageTarget->getSize(); LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY); LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]); if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) && fabs(intersection.data[1]) < (trackableSize.data[1] / 2)) { LOG("tapped inside the target!"); // do something here } tap = false; } ... } ... } // Edit existing method JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *, jobject) { ... // Cache the projection matrix: const QCAR::Tracker& tracker = QCAR::Tracker::getInstance(); const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration(); projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f, 2000.0f); // Cache the inverse projection matrix: inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix); }4. Make the following changes to ImageTargets.java:
private GestureDetector mGestureDetector; // Edit existing method protected void onCreate(Bundle savedInstanceState) { ... mGestureDetector = new GestureDetector(this, new TapGestureListener()); } @Override public boolean onTouchEvent(MotionEvent event) { if (mGestureDetector.onTouchEvent(event)) return true; else return false; } private native boolean nativeTapEvent(float tapX, float tapY); class TapGestureListener extends GestureDetector.SimpleOnGestureListener { @Override public boolean onSingleTapUp(MotionEvent ev) { nativeTapEvent(ev.getX(), ev.getY()); return true; } }- Kim
Re: How to get a tap event on the 3D model.
You need to make sure that you are using the modelview matrix for the correct target when using the code from the Dominoes sample. You may want to adapt the intersection methods to take the modelview matrix as an argument, rather than querying it globally.
- Kim
Re: How to get a tap event on the 3D model.
I could open a new topic but the issue is related with this tap method. Forget about 1st issue. I need tap event for multiple targets. How can I accomplish this by using this method. I implemented this method for one target succesfully; however, it does not work properly when it encountered by two or more target.
Re: How to get a tap event on the 3D model.
Does anyone has information about that?
Re: How to get a tap event on the 3D model.
I succesfully implemented tap event; however, there are two issue;
1 - When camera position is moved to up or down, as natural the target view is scaled in the real world and this event can detect that, before moving its height is for example, 400 unit, after moving it became 250 unit and this tap event can detect the change and the event is not invoked according to previous target position. But the problem is, there is no same issue about moving to right or left. When we move the camera to right or left, its real world view to the camera is changed but the event is invoked according to default width of the target. When we tap out of the target, it is logging inside the target.
2 - What about with multiple targets? When we try that implementation with two target, it is not processed. It is invoked when we tap somewhere else from targets.
Re: How to get a tap event on the 3D model.
i tried the same but during cygwin build getting error in ImageTarget.cpp---
1.getInstance is not member of qcar::Tracker.
2.qcar::Tracker has no member named getCameraCalibration.
can anyone help me with this?
Re: How to get a tap event on the 3D model.
The checkIntersectionLine function only works with cubes, it cannot be used to check for intersections against a mesh. You'll need to create a bounding box around your mesh to use that function. If you want finer-grained selection you'll have to find another solution, sorry!
- Kim
Re: How to get a tap event on the 3D model.
Hi Kim
Thanks for your help on marker tapping. Now, we are trying the object tapping on teapot. But getting some compilation errors in the following function.
We copied the code from Dominos for doing the object tapping instead of marker tapping.
bool
checkIntersectionLine(QCAR::Matrix44F transformA, QCAR::Vec3F pointA, QCAR::Vec3F pointB)
{
...
...
QCAR::Vec3F normalA1 = SampleMath::Vec3FTransformNormal(teapotNormals[0], transformA);
...
....
}
Q1. We are getting the following error
error: no matching function for call to 'SampleMath::Vec3FTransformNormal(float&, QCAR::Matrix44F&)'
: candidates are: static QCAR::Vec3F SampleMath::Vec3FTransformNormal(QCAR::Vec3F&, QCAR::Matrix44F&)
We removed the static keyword that was used in the declaration of teapotNormals
Q2. In the case of dominos it was a cube, we had only 9 values for dominosNOrmals. So we called the
function SampleMath::Vec3FTransformNormal 3 times.
But if we use some other object like a teapot, the number of values in the teapotNOrmals is NUM_TEAPOT_OBJECT_VERTEX * 3 ie (824*3). So
can you please let us know how to pass the n number of values of teapotNOrmals to the function SampleMath::Vec3FTransformNormal
Is it like we have to call function SampleMath::Vec3FTransformNormal 824 times in a loop?
Regards
gilesian
Re: How to get a tap event on the 3D model.
Sorry, that's outside the scope of what I can help with. You'll want to research 3D object picking, there are lots of approaches to this problem.
The Dominoes sample uses one approach, with object-aligned bounding boxes that surround the 3D models (you have to define these). See the checkIntersectionLine() method in Dominoes.cpp. "dominoNormals" and "dominoBaseVertices" just define a unit cube, and the "transformA" matrix is used to scale, rotate, and translate that cube to form a box that surrounds the model. Then it performs an intersection test with the line defined by "pointA" and "pointB". Trace through the code and see how that function is used, it might provide a good starting point.
- Kim
Re: How to get a tap event on the 3D model.
Yes, sorry, tapX and tapY should be floats (I've fixed the original post).
You would get NaN values if you forgot to set the global inverseProjMatrix or modelViewMatrix. Make sure you cache both of these.
// Cache the modelViewMatrix modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
- Kim
Re: How to get a tap event on the 3D model.
Hi Kim,
Thanks for the help..i implemented all the changes as u mentioned and we are able to get tap event but, we are unable to get the LinePlaneIntersection values, we are getting as NaN.
please let us know what exactly we are missing ...Thanks in advance
we change the data type of tapX and tapY variables to float in ImageTarget.cpp file..
Thanks
*Native tap coordinates (*Native screen space): 420.00, 168.00
***the x value420.0 (15717): the y value168.0
setPreviewCallbackFlag (pid 15717)
*point values from screentoplane**: 420.00, 168.00
**point values from screentoplane**: 0.05, 0.24
LinePLane Inetersection values*: LinePLane Inetersection values*: %.2f, %.2f, NaN,NaN
tap coordinates (screen space): 420.00, 168.00
tap coordinates (target space): NaN, NaN
The problem is that the x,y coordinates returned by a tap are in screen space. You'll want to project that to target space to determine whether or not a target was touched.
Here are the steps for determining whether a target was touched in the ImageTargets sample.
1. Copy SampleMath.h and SampleMath.cpp from Dominoes/jni to ImageTargets/jni.
2. Add SampleMath.cpp to the LOCAL_SRC_FILES variable in ImageTargets/jni/Android.mk.3. Make the following changes to ImageTargets.cpp:
#include "SampleMath.h" #include <QCAR/ImageTarget.h> #include <math.h> QCAR::Matrix44F inverseProjMatrix; QCAR::Matrix44F modelViewMatrix; bool tap = false; int tapX = 0; int tapY = 0; JNIEXPORT int JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_nativeTapEvent(JNIEnv *, jobject, jfloat x, jfloat y) { tapX = x; tapY = y; tap = true; } // Copy this from Dominoes.cpp bool linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd, QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection) { ... } // Copy this from Dominoes.cpp void projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd) { ... } // Edit existing method JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject) { ... // Did we find any trackables this frame? for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++) { // Get the trackable: const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx); //QCAR::Matrix44F modelViewMatrix = // QCAR::Tool::convertPose2GLMatrix(trackable->getPose()); // Cache the modelViewMatrix modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose()); if (tap) { QCAR::Vec3F intersection, lineStart, lineEnd; projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd); const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable); QCAR::Vec2F trackableSize = imageTarget->getSize(); LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY); LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]); if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) && fabs(intersection.data[1]) < (trackableSize.data[1] / 2)) { LOG("tapped inside the target!"); // do something here } tap = false; } ... } ... } // Edit existing method JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *, jobject) { ... // Cache the projection matrix: const QCAR::Tracker& tracker = QCAR::Tracker::getInstance(); const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration(); projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f, 2000.0f); // Cache the inverse projection matrix: inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix); }4. Make the following changes to ImageTargets.java:
private GestureDetector mGestureDetector; // Edit existing method protected void onCreate(Bundle savedInstanceState) { ... mGestureDetector = new GestureDetector(this, new TapGestureListener()); } @Override public boolean onTouchEvent(MotionEvent event) { if (mGestureDetector.onTouchEvent(event)) return true; else return false; } private native boolean nativeTapEvent(float tapX, float tapY); class TapGestureListener extends GestureDetector.SimpleOnGestureListener { @Override public boolean onSingleTapUp(MotionEvent ev) { nativeTapEvent(ev.getX(), ev.getY()); return true; } }- Kim
Re: How to get a tap event on the 3D model.
The problem is that the x,y coordinates returned by a tap are in screen space. You'll want to project that to target space to determine whether or not a target was touched.
Here are the steps for determining whether a target was touched in the ImageTargets sample.
1. Copy SampleMath.h and SampleMath.cpp from Dominoes/jni to ImageTargets/jni.
2. Add SampleMath.cpp to the LOCAL_SRC_FILES variable in ImageTargets/jni/Android.mk.
3. Make the following changes to ImageTargets.cpp:
#include "SampleMath.h" #include <QCAR/ImageTarget.h> #include <math.h> QCAR::Matrix44F inverseProjMatrix; QCAR::Matrix44F modelViewMatrix; bool tap = false; float tapX = 0; float tapY = 0; JNIEXPORT int JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_nativeTapEvent(JNIEnv *, jobject, jfloat x, jfloat y) { tapX = x; tapY = y; tap = true; } // Copy this from Dominoes.cpp bool linePlaneIntersection(QCAR::Vec3F lineStart, QCAR::Vec3F lineEnd, QCAR::Vec3F pointOnPlane, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection) { ... } // Copy this from Dominoes.cpp void projectScreenPointToPlane(QCAR::Vec2F point, QCAR::Vec3F planeCenter, QCAR::Vec3F planeNormal, QCAR::Vec3F &intersection, QCAR::Vec3F &lineStart, QCAR::Vec3F &lineEnd) { ... } // Edit existing method JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject) { ... // Did we find any trackables this frame? for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++) { // Get the trackable: const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx); //QCAR::Matrix44F modelViewMatrix = // QCAR::Tool::convertPose2GLMatrix(trackable->getPose()); // Cache the modelViewMatrix modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose()); if (tap) { QCAR::Vec3F intersection, lineStart, lineEnd; projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd); const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable); QCAR::Vec2F trackableSize = imageTarget->getSize(); LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY); LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]); if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) && fabs(intersection.data[1]) < (trackableSize.data[1] / 2)) { LOG("tapped inside the target!"); // do something here } tap = false; } ... } ... } // Edit existing method JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargets_startCamera(JNIEnv *, jobject) { ... // Cache the projection matrix: const QCAR::Tracker& tracker = QCAR::Tracker::getInstance(); const QCAR::CameraCalibration& cameraCalibration = tracker.getCameraCalibration(); projectionMatrix = QCAR::Tool::getProjectionGL(cameraCalibration, 2.0f, 2000.0f); // Cache the inverse projection matrix: inverseProjMatrix = SampleMath::Matrix44FInverse(projectionMatrix); }
4. Make the following changes to ImageTargets.java:
private GestureDetector mGestureDetector; // Edit existing method protected void onCreate(Bundle savedInstanceState) { ... mGestureDetector = new GestureDetector(this, new TapGestureListener()); } @Override public boolean onTouchEvent(MotionEvent event) { if (mGestureDetector.onTouchEvent(event)) return true; else return false; } private native boolean nativeTapEvent(float tapX, float tapY); class TapGestureListener extends GestureDetector.SimpleOnGestureListener { @Override public boolean onSingleTapUp(MotionEvent ev) { nativeTapEvent(ev.getX(), ev.getY()); return true; } }
- Kim
Re: How to get a tap event on the 3D model.
Hi kim ... so now what I'm planing to do is:
1. get x,y (in java) of the tap and pass it to native code
2. get the area of marker.
3. check if x,y lies in markers area.
Please tell me how can i achieve above thing .. also provide some code snippet ..
also i did not understand your 2nd suggestion "Or, you could use the projectScreenPointToPlane method from that sample to get the touch coordinates on the 2D marker plane." .. can u plz elaborate
Re: How to get a tap event on the 3D model.
hi kim ... im droping this ... because i feel its 2 difficult .. instead what ill try to find out is "Tap on marker"(on device screen). please provide me with some code snippet for the same ... Note: i have tried virtual button ... the problem with that is for virtual button we have to tap on the marker present on the paper and not on the device screen...
Re: How to get a tap event on the 3D model.
Handling tap events isn't built into the SDK. The Dominoes sample does have some code for selecting 3D objects on tap, but you'll have to dig into it a bit to reuse it. That sample intersects the touch against object-orientented bounding boxes (you have to define these per 3D object). Or, you could use the projectScreenPointToPlane method from that sample to get the touch coordinates on the 2D marker plane.
- Kim
The isTapOnScreenInsideTarget method is supposed to return true if the tap is in an area that covers the Image Target screen area (i.e. the projected area of the target rectangle onto the screen), not on a "3D model".
in Vuforia 4.0 you can see the same code written in Java.
If you need to tap on a 3D model, that's something that can be done at OpenGL and Android programming level (there is no Vuforia API dedicated to achieving that).