Log in or register to post comments

Detecting/Tracking Frame

May 30, 2011 - 9:53am #1

Hi!
I am new to Qualcomm and tried to built a project that only detects up to 5 Markers and tracking them. Is there anybody who can help me or has a very simple Project that has these features. I only want to track these markers and want to calculate the distance from the smartphone to the markers.

Re: Detecting/Tracking Frame

June 10, 2011 - 6:56am #28

I'm not sure why it matters if the markers are in front of or behind the camera... As you find markers, you can bring them into a unified coordinate system, choosing one marker as your world center. So let's say the first marker found is the origin (0, 0, 0). If you get two markers in view at once you can find the position of the second marker with the code I've given you. This only works if you're finding new markers while a known marker is in view, otherwise we have no knowledge of where the new marker is in relation to the ones we've seen.

The accelerometer can help you find the tilt of the phone, but it can't tell you in which compass direction the phone is facing. If your phone has a compass module you can use that.

- Kim

Re: Detecting/Tracking Frame

June 10, 2011 - 6:34am #27

Hi Kim!
I also want to get informations about saved markers(distance/direction) when they are in the back of the phone (6 o´clock). That´s why i asked for the accelerometer. Or do you think there is an easier way to get these informations of markers in the back of the phone. All markers were detected before they have to be calculated in the back. How i told before i want to build a global map, but at the moment i don´t know how to realize that with markers in the back.

Stephan

Re: Detecting/Tracking Frame

June 10, 2011 - 6:27am #26

Sure, you can use the accelerometer, but we don't have any guidelines for using this sensor data in conjunction with QCAR.

- Kim

Re: Detecting/Tracking Frame

June 10, 2011 - 3:11am #25

Hi Kim!

Thanks, stupid error.

Is it possible to use the Accelerometer in connection with the SDK?

Re: Detecting/Tracking Frame

June 9, 2011 - 6:27am #24

You have to scroll up in the log a bit, the error is there and quite clear:

E/AndroidRuntime( 9149): FATAL EXCEPTION: main
E/AndroidRuntime( 9149): java.lang.NullPointerException
E/AndroidRuntime( 9149): 	at com.qualcomm.QCARSamples.FrameMarkers.FrameMarkers.initApplicationAR(FrameMarkers.java:580)
E/AndroidRuntime( 9149): 	at com.qualcomm.QCARSamples.FrameMarkers.FrameMarkers.updateApplicationStatus(FrameMarkers.java:414)
E/AndroidRuntime( 9149): 	at com.qualcomm.QCARSamples.FrameMarkers.FrameMarkers.access$1(FrameMarkers.java:373)
E/AndroidRuntime( 9149): 	at com.qualcomm.QCARSamples.FrameMarkers.FrameMarkers$InitQCARTask.onPostExecute(FrameMarkers.java:148)
E/AndroidRuntime( 9149): 	at com.qualcomm.QCARSamples.FrameMarkers.FrameMarkers$InitQCARTask.onPostExecute(FrameMarkers.java:1)
E/AndroidRuntime( 9149): 	at android.os.AsyncTask.finish(AsyncTask.java:417)
E/AndroidRuntime( 9149): 	at android.os.AsyncTask.access$300(AsyncTask.java:127)
E/AndroidRuntime( 9149): 	at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:429)
E/AndroidRuntime( 9149): 	at android.os.Handler.dispatchMessage(Handler.java:99)
E/AndroidRuntime( 9149): 	at android.os.Looper.loop(Looper.java:144)
E/AndroidRuntime( 9149): 	at android.app.ActivityThread.main(ActivityThread.java:4937)
E/AndroidRuntime( 9149): 	at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime( 9149): 	at java.lang.reflect.Method.invoke(Method.java:521)
E/AndroidRuntime( 9149): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:868)
E/AndroidRuntime( 9149): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626)
E/AndroidRuntime( 9149): 	at dalvik.system.NativeStart.main(Native Method)
W/ActivityManager(  103):   Force finishing activity com.qualcomm.QCARSamples.FrameMarkers/.FrameMarkers
W/ActivityManager(  103): Activity pause timeout for HistoryRecord{462aae60 com.qualcomm.QCARSamples.FrameMarkers/.FrameMarkers}

This is telling you that you have a null pointer on line 580 of FrameMarkers.java. Indeed, you're using mRenderer before creating it.

- Kim

Re: Detecting/Tracking Frame

June 8, 2011 - 12:37pm #23

Hi Kim.

You think that also works if there is a saved marker behind the smartphone (6 o´clock), that i can recall the saved data and calculate the distance and direction?

I tried to understand the DDMS-Log because of the Sound that doesn´t work, but the only message is "activity destroy timeout" and "win death" and i spent hours with no success. How i told before there is no error in the Code, but the App is shut down after the splash-screen.

Perhaps you can check it? (http://ul.to/8x5yspj7)
I don´t know how to solve this problem.

Stephan

Re: Detecting/Tracking Frame

June 7, 2011 - 8:35pm #22

This post might have some code that is useful to you:

http://ar.qualcomm.at/node/2000410

It gets the position of one target in relation to another. It should be pretty easy to adapt for multiple targets, just choose one target as the world center.

- Kim

Re: Detecting/Tracking Frame

June 7, 2011 - 11:25am #21

Hi Kim,

ok, so i have to save spoken audiofiles with different distances and directions. Can i grab any coordinates from the marker and the camera? I try to build a global map, so i can recall marker/camera data and calculate the distance and direction from detected markers. I want to save detected markers and then calculate the new distance when i move in a room to the saved markers. Is that possible with coordinates i can grab?

Thank you very much for all your answers!!

Steph

Re: Detecting/Tracking Frame

June 7, 2011 - 6:37am #20

We won't be able to help with APIs for text-to-speech, this is too domain-specific. As for why that code isn't working for you, I suggest getting to know the Eclipse debugger to set breakpoints and step through the Java code. Also look at the DDMS perspective in Eclipse for viewing the device log, if the application crashes helpful messages are often printed there.

- Kim

Re: Detecting/Tracking Frame

June 7, 2011 - 6:33am #19

If you change the method signature in Java, you must also change the JNI types in this line of code in native:

jmethodID method = env->GetMethodID(javaClass, "reportDistance", "(F)V");

(F)V - a method that takes a float and returns void

http://java.sun.com/docs/books/jni/html/types.html

- Kim

Re: Detecting/Tracking Frame

June 7, 2011 - 2:36am #18

Ihttp://ar.qualcomm.at/node/2000313havehttp://ar.qualcomm.at/node/2000313alsohttp://ar.qualcomm.at/node/2000313anotherhttp://ar.qualcomm.at/node/2000313questionhttp://ar.qualcomm.at/node/2000313:)http://ar.qualcomm.at/node/2000313Whathttp://ar.qualcomm.at/node/2000313ishttp://ar.qualcomm.at/node/2000313thehttp://ar.qualcomm.at/node/2000313easiesthttp://ar.qualcomm.at/node/2000313wayhttp://ar.qualcomm.at/node/2000313tohttp://ar.qualcomm.at/node/2000313speakhttp://ar.qualcomm.at/node/2000313outhttp://ar.qualcomm.at/node/2000313thehttp://ar.qualcomm.at/node/2000313distance.http://ar.qualcomm.at/node/2000313Ishttp://ar.qualcomm.at/node/2000313thehttp://ar.qualcomm.at/node/2000313onlyhttp://ar.qualcomm.at/node/2000313posibilityhttp://ar.qualcomm.at/node/2000313tohttp://ar.qualcomm.at/node/2000313speakhttp://ar.qualcomm.at/node/2000313outhttp://ar.qualcomm.at/node/2000313stringshttp://ar.qualcomm.at/node/2000313withhttp://ar.qualcomm.at/node/2000313TextToSpeechhttp://ar.qualcomm.at/node/2000313orhttp://ar.qualcomm.at/node/2000313ishttp://ar.qualcomm.at/node/2000313ithttp://ar.qualcomm.at/node/2000313betterhttp://ar.qualcomm.at/node/2000313tohttp://ar.qualcomm.at/node/2000313usehttp://ar.qualcomm.at/node/2000313anotherhttp://ar.qualcomm.at/node/2000313posibility.http://ar.qualcomm.at/node/2000313Ihttp://ar.qualcomm.at/node/2000313triedhttp://ar.qualcomm.at/node/2000313tohttp://ar.qualcomm.at/node/2000313playhttp://ar.qualcomm.at/node/2000313ahttp://ar.qualcomm.at/node/2000313soundfilehttp://ar.qualcomm.at/node/2000313whenhttp://ar.qualcomm.at/node/2000313ahttp://ar.qualcomm.at/node/2000313markerhttp://ar.qualcomm.at/node/2000313ishttp://ar.qualcomm.at/node/2000313detected,http://ar.qualcomm.at/node/2000313likehttp://ar.qualcomm.at/node/2000313muhammadgyanhttp://ar.qualcomm.at/node/2000313posted.
http://ar.qualcomm.at/node/2000313->http://ar.qualcomm.at/node/2000313https://ar.qualcomm.com/arforums/showthread.php?p=2612&postcount=11
Therehttp://ar.qualcomm.at/node/2000313ishttp://ar.qualcomm.at/node/2000313nohttp://ar.qualcomm.at/node/2000313error,http://ar.qualcomm.at/node/2000313buthttp://ar.qualcomm.at/node/2000313thehttp://ar.qualcomm.at/node/2000313apphttp://ar.qualcomm.at/node/2000313ishttp://ar.qualcomm.at/node/2000313shuthttp://ar.qualcomm.at/node/2000313downhttp://ar.qualcomm.at/node/2000313afterhttp://ar.qualcomm.at/node/2000313thehttp://ar.qualcomm.at/node/2000313samplehttp://ar.qualcomm.at/node/2000313Splahhttp://ar.qualcomm.at/node/2000313Screen.http://ar.qualcomm.at/node/2000313Sohttp://ar.qualcomm.at/node/2000313ihttp://ar.qualcomm.at/node/2000313can´thttp://ar.qualcomm.at/node/2000313testhttp://ar.qualcomm.at/node/2000313ifhttp://ar.qualcomm.at/node/2000313thehttp://ar.qualcomm.at/node/2000313Codehttp://ar.qualcomm.at/node/2000313works.

Re: Detecting/Tracking Frame

June 7, 2011 - 2:10am #17

Hi Kim!

You told me how to use the distance in Java. Your Code works, but when i change the returntype in your examplefunction, the app shut down after a marker is detected. Any mistake in the nativecode?

 public /**float*/ void reportDistance(float distance)
{
    DebugLog.LOGD("Distance: " + distance);
    
    //return distance;
    // do other stuff with the distance in Java
}

Re: Detecting/Tracking Frame

June 6, 2011 - 6:56am #16

You can just use a different variable name for the camera position :)

My last post gave you the camera position and the direction it is looking in (as a vector). The camera position may be enough for your purposes, pull out the x and y values and do some math to find the angle off the twelve-o'clock position.

- Kim

Re: Detecting/Tracking Frame

June 6, 2011 - 3:00am #15

Hi Kim!

Thanks a lot. One problem is, that it is not possible to declarate "QCAR::Vec3F position" again, after i declared it for the distance.

  for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
    {
        // Get the trackable:
        const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
        QCAR::Matrix44F modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(trackable->getPose());        
      
        QCAR::Vec3F position(modelViewMatrix.data[12], modelViewMatrix.data[13], modelViewMatrix.data[14]);
        float distance = sqrt(position.data[0] * position.data[0] +
                              position.data[1] * position.data[1] +
                              position.data[2] * position.data[2]);
        LOG("distance: %f", distance);

           jclass javaClass = env->GetObjectClass(obj);
           jmethodID method = env->GetMethodID(javaClass, "reportDistance", "(F)V");
           env->CallObjectMethod(obj, method, distance);

           QCAR::Matrix44F inverseModelView = SampleMath::Matrix44FTranspose(SampleMath::Matrix44FInverse(modelViewMatrix));

           QCAR::Vec3F direction(inverseModelView.data[8], inverseModelView.data[9], inverseModelView.data[10]);
   ERROR    // QCAR::Vec3F position(inverseModelView.data[12], inverseModelView.data[13], inverseModelView.data[14]);
           LOG("camLookAt: %f, %f, %f", direction.data[0], direction.data[1], direction.data[2]);
   ERROR   // LOG("camPosition: %f, %f, %f", position.data[0], position.data[1], position.data[2]);

In need both, direction and distance. How can i interprate the vector? My goal is that when a marker is detected the application speak out the distance and the direction of the detected marker (in time-> e.g. table on 11 o´clock).

Stephan

Re: Detecting/Tracking Frame

June 5, 2011 - 12:17pm #14

For the direction, one approach is to take the inverse of the modelview matrix, which again gets you information about the camera's position and orientation in relation to a target at the world origin. With this particular matrix, you can pull interesting vectors directly from the columns. The third column is a lookAt vector, and the fourth column is the camera's position.

QCAR::Matrix44F inverseModelView = SampleMath::Matrix44FTranspose(SampleMath::Matrix44FInverse(modelViewMatrix));
QCAR::Vec3F direction(inverseModelView.data[8], inverseModelView.data[9], inverseModelView.data[10]);
QCAR::Vec3F position(inverseModelView.data[12], inverseModelView.data[13], inverseModelView.data[14]);
LOG("camLookAt: %f, %f, %f", direction.data[0], direction.data[1], direction.data[2]);
LOG("camPosition: %f, %f, %f", position.data[0], position.data[1], position.data[2]);

You'll need to grab the SampleMath.h and SampleMath.cpp files from the Dominoes project, and add SampleMath.cpp to your project's Android.mk file (to the LOCAL_SRC_FILES variable).

For your second question, just change the method signature, those variables are passed in:

Java_com_qualcomm_QCARSamples_FrameMarkers_FrameMarkersRenderer_renderFrame(JNIEnv *env, jobject obj)

- Kim

Re: Detecting/Tracking Frame

June 5, 2011 - 6:48am #13

I have also another question to get the distance from the native code to Java.
I put your expample after my function for the distance and got two errormessages because env and obj is not declared in this scope. The distance is only declared in this part, so i thought i have to put it after your code for the distance!?

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_FrameMarkers_FrameMarkersRenderer_renderFrame(JNIEnv *, jobject)
{
...

  QCAR::Matrix44F modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(trackable->getPose());        
      

        QCAR::Vec3F position(modelViewMatrix.data[12], modelViewMatrix.data[13], modelViewMatrix.data[14]);
         float distance = sqrt(position.data[0] * position.data[0] +
                              position.data[1] * position.data[1] +
                              position.data[2] * position.data[2]);
       
        
             jclass javaClass = env->GetObjectClass(obj);
             jmethodID method = env->GetMethodID(javaClass, "reportDistance", "(F)V");
             env->CallObjectMethod(obj, method, distance);
....

Re: Detecting/Tracking Frame

June 4, 2011 - 2:29am #12

Hi Kim!

With Refresh it works :) Your code for the distance also works. Is there any function for the direction from the smartphone to the marker?

Thanks, Stephan

Re: Detecting/Tracking Frame

June 1, 2011 - 7:19am #11

Anything logged with the native LOG method or the Java DebugLog methods should show up in the log cat.

Are you building your native code using the "ndk-build" command? You may need to refresh the Eclipse project afterwards to grab the changes, right-click on the project in the Package Explorer and choose Refresh.

- Kim

Re: Detecting/Tracking Frame

June 1, 2011 - 1:46am #10

When i have one of the Markers in the Stream, there is no DistanceData in the LOG. The only messages i get are:

06-01 10:31:50.037: INFO/BatteryStatsImpl(98): notePhoneSignalStrengthLocked: 4->3
06-01 10:31:59.037: INFO/BatteryStatsImpl(98): notePhoneSignalStrengthLocked: 3->4
06-01 10:32:00.057: ERROR/ClockWidget(259): weatherClock onReceive~ mUseAnimation:false
06-01 10:32:00.057: ERROR/ClockWidget(259): weatherClock onReceive~ mUseAnimation:false
...
The information about the distance should be in the LogCat or not? Is it necessary to do any configurations to get the information in the LogCat?

Thanks a lot, Stephan

Re: Detecting/Tracking Frame

May 31, 2011 - 7:30pm #9

Yes, the distance is logged in real-time when the marker is visible.

To get the distance from native to Java you use the JNI (Java Native Interface). For example:

FrameMarkers.cpp

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_FrameMarkers_FrameMarkersRenderer_renderFrame(JNIEnv *env, jobject obj)
{
    ...
    jclass javaClass = env->GetObjectClass(obj);
    jmethodID method = env->GetMethodID(javaClass, "reportDistance", "(F)V");
    env->CallObjectMethod(obj, method, distance);
    ...
}

FrameMarkersRenderer.java

public void reportDistance(float distance)
{
    DebugLog.LOGD("Distance: " + distance);
    // do other stuff with the distance in Java
}

- Kim

Re: Detecting/Tracking Frame

May 31, 2011 - 1:12pm #8

The information is logged when a marker is recognized in the stream, is that right? Another question i have is how i can connect this with TextToSpeech? I want the distance to be spoken when it is recognized. But how to connect the TextToSpeech-Library with the distance from the native Code.

Steph

Re: Detecting/Tracking Frame

May 31, 2011 - 12:07pm #7

What application are you viewing the .cpp code in? Depending on how Eclipse is set up it may show those warnings. What really matters are the results of running ndk-build, however. Be sure to run this every time you change the native code.

This bit of the code should go in the FrameMarkers.cpp renderFrame method, right after the modelViewMatrix is defined:

QCAR::Vec3F position(modelViewMatrix.data[12], modelViewMatrix.data[13], modelViewMatrix.data[14]);
float distance = sqrt(position.data[0] * position.data[0] +
                      position.data[1] * position.data[1] +
                      position.data[2] * position.data[2]);
LOG("distance: %f", distance);

The output ends up in the Android log file. In Eclipse, you can open the DDMS perspective to view this log. Or you can run "adb logcat" from the command line (e.g. Cygwin).

- Kim

Re: Detecting/Tracking Frame

May 31, 2011 - 5:33am #6

Hi Kim!
I now tested the FrameMarker-SampleProject. I tried to your Codesnippet for the distance. Is it right that i have to put it in the FrameMarkers.cpp? In this .cpp there are 2 errors -> Multiple Markers at this line

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_FrameMarkers_FrameMarkers_setActivityPortraitMode(JNIEnv *, jobject, jboolean isPortrait)
{ ....

and

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_FrameMarkers_FrameMarkersRenderer_renderFrame(JNIEnv *, jobject)
{ ....

the rest (...) is yellow underlined. The Project works fine with these errors, but would be nice to know why :)

When the distance of the Markers was calculated, how can i get the logged distance? cout in the mainActivity? Sorry, i have little experience in C++.

Re: Detecting/Tracking Frame

May 31, 2011 - 12:34am #5

Generally i want to use those markers which can still be detected from a long distance and can be detected simultaneously in a stream. Can you say that a certain marker can be better detected from a long distance.

Stephan

Re: Detecting/Tracking Frame

May 30, 2011 - 2:49pm #4

The sample projects are the right starting point, I don't suggest starting with an empty Android project. It sounds like you want to track frame markers, not image targets, is that right? If so, definitely start with the FrameMarkers sample.

- Kim

Re: Detecting/Tracking Frame

May 30, 2011 - 2:29pm #3

I only tested the ImageTargets-SampleProject that renders the Teapots on the pictures, that works nice. I didn´t recognized the other Sampleproject you mentioned. Hope it will help me more. Before i can test the Code you posted me, i have to create the Frame i told before. I didn´t succeed in creating such a project, so i had no chance to test your code. Is there any easy tutorial getting started with things like that, or is the only possibilty to check the SampleCodes? Thanks a lot for your fast reply.

Re: Detecting/Tracking Frame

May 30, 2011 - 1:39pm #2

Have you tried starting with the FrameMarkers sample? It tracks four different frame markers. The renderFrame method in FrameMarkers.cpp would be a good starting point.

Did you have any luck with the distance code I gave you before?

http://ar.qualcomm.at/node/2000437

- Kim

Log in or register to post comments