Log in or register to post comments

How to drag the 3D model

March 24, 2013 - 8:02am #1

Hi, I want to drag the 3D model with onTouchEvent method. I get the screen coordinate, and then pass those values to translate the object. But, since I did the translation directly to the modelview matrix , my 3D model get translated in clip coordinate which is I did not expect. So, How to make it moves according to screen coordinate? and how to pass the screen coordinate to the translations.

Thanks

How to drag the 3D model

July 22, 2013 - 6:25am #23

All the functions in ImageTargets.cpp are good examples of C++ functions that are called from Java;

for example the function called "_renderFrame()" is called from the class ImageTargetsRenderer.java (by calling "renderFrame()" as you can see in the code).

The mechanism you have to use is the same for any function that you want to call from Java, i.e.:

  • declare the method as a "native" one in Java
  • implement the method in native code (similarly to the _renderFrame() method).

If you are still in doubt, the best is to familiarize with JNI first (note: this Forum is not the right place where to learn JNI)

 

How to drag the 3D model

July 22, 2013 - 5:14am #22

Can you give me an example to call SampleUtil::rotatePoseMatrix() in ImageTargets.java file? Thank you very much.

How to drag the 3D model

July 22, 2013 - 5:12am #21
In opengl, i know how to call this rotation function in Java file
gl.glRotatef(xrot, 1.0f, 0.0f, 0.0f);
gl.glRotatef(yrot, 0.0f, 1.0f, 0.0f);
 
But for vuforia, how should call the similar function in Java file? Can you give me an example? 

How to drag the 3D model

July 22, 2013 - 5:08am #20

Yes, you can call any C++ method from Java via JNI.

How to drag the 3D model

July 22, 2013 - 4:10am #19

Thank you. If i want to call SampleUtils::rotatePoseMatrix() from Java, am i able to do that?

 

How to drag the 3D model

July 22, 2013 - 3:55am #18

JNI interaction (Java to C++ and C++ to Java) is explained here:

https://developer.vuforia.com/resources/dev-guide/java-native-interface

 

How to drag the 3D model

July 22, 2013 - 2:20am #17

How should i pass the value from Java to C++? I am not sure how to achieve this point.

How to drag the 3D model

June 21, 2013 - 12:10am #16

You need to intercept the touch events in Java (not in C++!); 

see this article from the Android SDK developer guide:http://developer.android.com/training/graphics/opengl/touch.html

Then, you will need to:

- compute the amount of drag (in pixels) in Java, in X and Y direction;

- pass those values to C++ using JNI (where you can then map the values to the screenDeltaX and screenDeltaY variables).

 

How to drag the 3D model

June 20, 2013 - 9:46pm #15

"You then need to implement the touch event handling in Java and pass the "screenDeltaX" and "screenDeltaY" to C++" 

Could you explain more please ? I need to finish this project at night :(

Now I am trying to tap on 3d model but in this regard have not been successful yet.

Edit: I can tap the screen  ( helped me this link https://developer.vuforia.com/forum/android/how-get-tap-event-3d-model#comment-2028743 )

// Do you want to say in this direction? I mean Am I on the right way? to drag my 3d model.
 
Now What should I do to drag my 3d model ? 
if (tap)
        {
            QCAR::Vec3F intersection, lineStart, lineEnd;
            projectScreenPointToPlane(QCAR::Vec2F(tapX, tapY), QCAR::Vec3F(0, 0, 0), QCAR::Vec3F(0, 0, 1), intersection, lineStart, lineEnd);

            const QCAR::ImageTarget* imageTarget = static_cast<const QCAR::ImageTarget*>(trackable);
            QCAR::Vec2F trackableSize = imageTarget->getSize();

            LOG("tap coordinates (screen space): %.2f, %.2f", tapX, tapY);
            LOG("tap coordinates (target space): %.2f, %.2f", intersection.data[0], intersection.data[1]);

            if (fabs(intersection.data[0]) < (trackableSize.data[0] / 2) &&
                fabs(intersection.data[1]) < (trackableSize.data[1] / 2))
            {
                LOG("tapped inside the target!");
                // do something here ?
            }

            tap = false;
        }

 

 

 

How to drag the 3D model

June 19, 2013 - 10:53pm #14

You then need to implement the touch event handling in Java and pass the "screenDeltaX" and "screenDeltaY" to C++

 

How to drag the 3D model

June 19, 2013 - 8:55pm #13

Thanks for response AlessandroB 

I have no error but doesnt drag 3d model.  What I am missing :(

Edit: Could anyone help me pls ? this is for my graduate thesis.

How to drag the 3D model

June 19, 2013 - 12:08pm #12

fabs is a function defined in math.h;

You need to add this line:

#include <math.h>

 

How to drag the 3D model

June 19, 2013 - 11:59am #11

Hi guys 

I am trying to drag my 3d model by touch 

I followed this link to do this https://developer.vuforia.com/forum/faq/opengl-how-can-i-move-my-3d-model-screen-plane

  • copied SampleMath.ccp and SampleMath.h in my jni folder from Dominoes Sample.
  • Located in Android.mk
  • In ImageTargets.ccp added QCAR::Vec3F targetCumulatedDisplacement(0.0f, 0.0f, 0.0f); as global.
  • Added this method before the renderFrame() function:
void
computeTargetTranslationFromScreenVector(float screenDeltaX, float screenDeltaY, 
        QCAR::Matrix44F & modelViewMatrix, QCAR::Vec3F & result)
{
    QCAR::Vec3F screenAlignedDisplacement;
    screenAlignedDisplacement.data[0] = screenDeltaX;
    screenAlignedDisplacement.data[1] = screenDeltaY;
    screenAlignedDisplacement.data[2] = 0.0f;
 
    // Compute matrix to pass from Eye Coordinates to Object Coordinates
    QCAR::Matrix44F inverseModelViewMatrix = 
                   SampleMath::Matrix44FInverse(modelViewMatrix);
 
    // Convert the screen-aligned displacement vector to Object Coordinates
    // (i.e. in the target local reference frame)
    QCAR::Vec3F localTargetDisplacement = SampleMath::Vec3FTransformNormal(
                              screenAlignedDisplacement, inverseModelViewMatrix);
 
    // Compute a speed factor based on the distance of the target from the camera
    float distanceObjectToCamera = fabs(modelViewMatrix.data[14]);
    float speed = 0.001f * distanceObjectToCamera;// TODO adjust value to your needs
 
    // set the result taking the speed factor into account
    result.data[0] = speed * localTargetDisplacement.data[0];
    result.data[1] = speed * localTargetDisplacement.data[1];
    result.data[2] = speed * localTargetDisplacement.data[2];
}
  • In renderFrame() method 
QCAR::Matrix44F modelViewProjection;
 
SampleUtils::translatePoseMatrix(0.f, 0.f, kObjectScale,
                                 &modelViewMatrix.data[0]);
SampleUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale,
                                 &modelViewMatrix.data[0]);
 
 
float screenDeltaX = 2.0f;// moving to the right 2 pixels every frame
float screenDeltaY = 0.0f;//
 
QCAR::Vec3F targetLocalDisplacement;
computeTargetTranslationFromScreenVector(screenDeltaX, screenDeltaY, 
                             modelViewMatrix, targetLocalDisplacement);
 
 
targetCumulatedDisplacement.data[0] += targetLocalDisplacement.data[0];
targetCumulatedDisplacement.data[1] += targetLocalDisplacement.data[1];
targetCumulatedDisplacement.data[2] += targetLocalDisplacement.data[2];
 
 
SampleUtils::translatePoseMatrix(targetCumulatedDisplacement.data[0],
                targetCumulatedDisplacement.data[1],
                targetCumulatedDisplacement.data[2],
                                 &modelViewMatrix.data[0]);
 
SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                            &modelViewMatrix.data[0],
                            &modelViewProjection.data[0]);

but I have got some errors. .

Cygwin 
------------------------
$ ndk-build
Compile++ arm    : AugmentedRealityTest <= ImageTargets.cpp
jni/ImageTargets.cpp: In function 'void computeTargetTranslationFromScreenVector(float, float, QCAR::Matrix44F&, QCAR::Vec3F&)':
jni/ImageTargets.cpp:298:65: error: 'fabs' was not declared in this scope
/cygdrive/c/Development/Android/android-ndk-r8e/build/core/build-binary.mk:272: recipe for target `obj/local/armeabi/objs/AugmentedRealityTest/ImageTargets.o' failed
make: *** [obj/local/armeabi/objs/AugmentedRealityTest/ImageTargets.o] Error 1
-----------------------------
 
 
Anyone help me pls ?

How to drag the 3D model

March 30, 2013 - 2:42am #10

Hi, 

I would suggest to Log the vqlue of the targetTranslation global variable,

it might help just resetting it to 0,0,0 in the initialization function of ImageTargets.cpp

How to drag the 3D model

March 30, 2013 - 12:48am #9

Hi, I just realised that after moving my 3D model on certain positions, it did not return to it's original position as I destroyed and running my  app again. In my logcat, the dataset has been destroyed, but I could not figure out why it stayed on the latest position I moved. Do you know how to solve it?

 

How to drag the 3D model

March 25, 2013 - 5:49am #8

You're welcome.

The reason why you need to swap the axis is because the XYZ reference frame which is attached to the camera is still following the "landscape" camera layout (i.e. as you correctlty say, our video mode is still landscape, even though the screen orientation is portrait).

 

How to drag the 3D model

March 25, 2013 - 5:20am #7

Hi, thank you for being friendly  helping me to solve this issue. Glad to see it works. However, I still curious about the swaping deltaX and deltaY, why it must be swapped  when run in protrait mode? is that due to our videoMode is landscape?

thanks

How to drag the 3D model

March 25, 2013 - 3:26am #6

Hi,

I have prepared a step-by-step guide to do what you are trying to achieve, please check here and compare with your current implementation:

https://developer.vuforia.com/forum/faq/opengl-how-can-i-move-my-3d-model-screen-plane

This should also answer your latest questions.

 

How to drag the 3D model

March 25, 2013 - 3:12am #5

Hi, thanks it's clearer now. Sorry for late reply because I'm facing problem due to applying those codes you gave. 
Problem 1, why when I stopped dragging  my model, it return to it's original position?
Problem 2, why when I dragged in Y+ my 3d model moved to X+ axis and vice versa?

Another issue, can I just use the delta screen motion x and y , and use those values to translate the modelView? because normal and tangent vector are both unaffected by translation. Please correct me If I misunderstood.

Thanks

How to drag the 3D model

March 24, 2013 - 11:33pm #4

Hi, transformNormal is a function that transform a direction vector from one reference frame to another; in your case it should transform the direction vector that represents your displacement in pixels into a 3D vector that represents a translation of your target in its local reference frame.

Note: you can find the transformNormal function in SampleMath.h and SampleMath.cpp in the Dominoies sample (so you will need to copy those files to your project) 

After getting the 3D vector converted from screen motion, you can use it in your renderFrame function like in this code:

SampleUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale,
                                         &modelViewMatrix.data[0]);

SampleUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale,
                                     &modelViewMatrix.data[0]);

SampleUtils::translatePoseMatrix(targetTranslation.data[0], targetTranslation.data[1], targetTranslation.data[2],
                                     &modelViewMatrix.data[0]);

SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                                    &modelViewMatrix.data[0] ,
                                    &modelViewProjection.data[0]);

Note: you might need to adjust the "speed" factor in the function of my previous post if the motion is either too slow or too fast

I hope it is clearer now.

 

How to drag the 3D model

March 24, 2013 - 8:56pm #3

Hi, thanks for your helps. But before I applied these code, I would like to ask for things that I've not understood yet.
 

1. What does the transformNormal do? and for what purpose we do transformNormal?

2. After getting 3D vector converted from screen motion (target translation), where I should pass these values? 

Sometimes it would be helpful if you explain in such a way you explain to a beginner to avoid misunderstanding.

Thank you for sharing. 

How to drag the 3D model

March 24, 2013 - 3:31pm #2

Hi, you need to convert your screen motion to a 3D vector in the local target reference frame (which you can then apply as a translation);

to compute the 3D vector that corresponds to your screen motion, you could use a function like this:

void
projectScreenMotion(float delta_x_pixels, float delta_y_pixels, QCAR::Vec3F & targetTranslation)
{
    // Eye Coordinates to Object Coordinates
    QCAR::Matrix44F inverseModelViewMatrix = SampleMath::Matrix44FInverse(modelViewMatrix);

    float speed = 1.0f;// TODO find good value for speed
    QCAR::Vec3F vectorDelta;
    vectorDelta.data[0] = speed * delta_x_pixels;
    vectorDelta.data[1] = speed * delta_y_pixels;

    QCAR::Vec3F localDelta = SampleMath::Vec3FTransformNormal(vectorDelta, inverseModelViewMatrix);
    targetTranslation.data[0] = localDelta.data[0];
    targetTranslation.data[1] = localDelta.data[1];
    targetTranslation.data[2] = localDelta.data[2];
}

Note the use of the modelview inverse.

Log in or register to post comments