Android/iOS - How can I update my UI in response to tracking events?

August 1, 2012 - 11:55am #1

The Vuforia samples handle tracking events in the native renderFrame method, which is called on the GL thread. In order to make changes to the Android UI in response to a tracking event, two concepts must be used: JNI (Java Native Interface) calls from native code to Java, and cross-thread calls from the GL thread to the main thread. The latter is required because the Android UI can only be modified from the main (UI) thread.

For more on the JNI see this article: [JNI Article]

For more on threading see this article: [Threading Article]

The following code snippets show how to display an Android Toast message when a target is first found. Start with the ImageTargets sample and make the following changes to ImageTargets.cpp:

 

int lastTrackableId = -1;

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv* env, jobject obj)
{
    ...

    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
    {
        // Get the trackable:
        const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);

        // Compare this trackable's id to a globally stored id
        // If this is a new trackable, find the displayMessage java method and
        // call it with the trackable's name
        if (trackable->getId() != lastTrackableId) {
            jstring js = env->NewStringUTF(trackable->getName());
            jclass javaClass = env->GetObjectClass(obj);
            jmethodID method = env->GetMethodID(javaClass, "displayMessage", "(Ljava/lang/String;)V");
            env->CallVoidMethod(obj, method, js);
            lastTrackableId = trackable->getId();
        }

        ...
    }

    ...
}

This code uses JNI calls to find the displayMessage method of the ImageTargetsRenderer Java class. Note the "env" and "obj" variables are passed in as arguments to the renderFrame method. The lastTrackableId global variable is used to ensure the message is only displayed once when the target becomes active.

 

Next, add the following to ImageTargetsRenderer.java:

    // A handler object for sending messages to the main activity thread
    public static Handler mainActivityHandler;

    // Called from native to display a message
    public void displayMessage(String text)
    {
        // We use a handler because this thread cannot change the UI
        Message message = new Message();
        message.obj = text;
        mainActivityHandler.sendMessage(message);
    }

This is the method that the native code will call when the trackable becomes active. This method will be called from the GL thread, and uses a Handler to queue up a message on the main (UI) thread.

Finally, add the following to ImageTargets.java:

    protected void onResume()
    {
        super.onResume();

        // Create a new handler for the renderer thread to use
        // This is necessary as only the main thread can make changes to the UI
        ImageTargetsRenderer.mainActivityHandler = new Handler() {
            @Override
            public void handleMessage(Message msg) {
                Context context = getApplicationContext();
                String text = (String) msg.obj;
                int duration = Toast.LENGTH_SHORT;
                Toast toast = Toast.makeText(context, text, duration);
                toast.show();
            }
        };

        ...
    }

Here we create a new Handler from the onResume method, which runs on the main (UI) thread, and assign it to the public variable we added to the ImageTargetsRenderer class. When this Handler receives a message it displays a Toast on the screen with the included message (a Toast is a notification that is overlaid on the screen for a few seconds). The Dominoes sample includes more code for changing the UI (e.g. showing/hiding buttons) from native code.

 

Topic locked