Actually, adding that script to an empty gameobject won't work. Adding it to the ARCamera does. Binding a material doesn't make a difference. I agree with you that it shouldn't make a difference where you attach the script. But it does. I don't know why.

Edit:
More on a unified coordinate system here:
http://ar.qualcomm.at/node/2001030

That should work, I added my script to the ARCamera but that shouldn't make a difference.

Perhaps you need to create and bind a material. See the sample code here:

http://unity3d.com/support/documentation/ScriptReference/GL.html

- Kim

I cannot get that code to work. Can you provide some more information on how to get that to work? Where do I attach the script to? What objects do I drop on onto the inspector public variables?

I Added two frame marker prefabs to the Hierarchy. I created an empty game object, and attached your script to that. I dropped the prefabs from the Hierarchy view to the variable slots of the connect trackables script. That doesn't work...

No need for this in Unity, the trackables are already brought into a unified coordinate system to work with the Unity game engine. You can refer to the transform.position values directly. So, to do something similar:

public class ConnectTrackables : MonoBehaviour
{
    public TrackableBehaviour m_Trackable1;
    public TrackableBehaviour m_Trackable2;
    
    void OnPostRender()
    {
        if ((m_Trackable1.CurrentStatus == TrackableBehaviour.Status.DETECTED ||
             m_Trackable1.CurrentStatus == TrackableBehaviour.Status.TRACKED) &&
            (m_Trackable2.CurrentStatus == TrackableBehaviour.Status.DETECTED ||
             m_Trackable2.CurrentStatus == TrackableBehaviour.Status.TRACKED))
        {
            Vector3 pos1 = m_Trackable1.transform.position;
            Vector3 pos2 = m_Trackable2.transform.position;
            
            GL.Begin(GL.LINES);
            GL.Color(new Color(1,0,0,1));
            GL.Vertex3(pos1.x, pos1.y, pos1.z);
            GL.Vertex3(pos2.x, pos2.y, pos2.z);
            GL.End();
        }
    }
}

(sorry, I'm more familiar with C#)

- Kim

Kim, can you translate this into code for unity? Preferably into .js
I am not sure how to convert all this C++ matrix stuff into unity script.
Thanks.

Thanks a ton! I've got my project working perfectly. It first draws a red line between each trackable, in the order that they are picked up. When I click Nearest Neighbor in the menu it switches the line to the sorted order and turns blue. It's pretty awesome watching Nearest Neighbor change its result in real time as you move an object around. I've had it tracking up to seven targets, which is about as many as I can fit on screen at a time, with the targets being around 6"x4".

Finally, you asked about the various OpenGL arrays:

vertices[] - These are the 3D points that define your geometry. This array is always needed.

texcoords[] - These are 2D U,V texture coordinates, one per vertex. They describe how to map a 2D image onto a 3D object. All of our samples use textures to color the objects, for example look at the teapot textures in the ImageTargets/assets folder. This array is only needed when you are rendering with textures, sometimes you may prefer rendering with a solid color.

normals[] - These point outward from each vertex of the model, perpendicular to the surface. They are used for lighting calculations. Our samples include them but do not use them, feel free to disable this array.

indices[] - Sometimes all the data in the arrays above can be read in consecutive order (e.g each three values might define a triangle). Sometimes however, the values are not in order and you need an array of indices to describe how to render each primitive from the arrays. This allows you to reuse vertices and save space. If you have indices, you use glDrawElements. If you don't, you use glDrawArrays.

- Kim

Okay, so that code is somewhat complex, because this is a complicated problem! The problem is that the pose matrix defines a local coordinate system for each trackable, with the center of the trackable as the origin (0, 0, 0). That makes it really easy to draw objects on top of each trackable independently, but not so easy to draw content that spans multiple trackables (taking their relative orientation into account).

The solution is to pick one trackable to act as the world center, and bring all other trackables into its coordinate system. We do that by multiplying the inverse pose of our world center target (A) by the pose our other targets (B). This creates an offset matrix that can be used to bring points on B into A's coordinate system. See the code above.

Now, we can bind a single modelview matrix tied to our world center and render all the points from the world center's point of view. That should let you draw lines between targets.

- Kim

Let me start with a post including some code that should be helpful. The next post will explain some things :)

This is a replacement renderFrame method for the ImageTargets sample. It will render a line connecting the center of the chips target to the center of the stones target. You will need to do a few setup steps first:

1) Copy SampleMath.cpp and SampleMath.h from the Dominoes/jni folder to the ImageTargets/jni folder.
2) Add SampleMath.cpp to the LOCAL_SRC_FILES flag in the ImageTargets Android.mk file.
3) Uncomment the following lines in ImageTargets.cpp:

QCAR::setHint(QCAR::HINT_MAX_SIMULTANEOUS_IMAGE_TARGETS, 2);
QCAR::setHint(QCAR::HINT_IMAGE_TARGET_MULTI_FRAME_ENABLED, 1);

And here's the renderFrame method:

#include "SampleMath.h"

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *, jobject)
{
    // Clear color and depth buffer 
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
    
    // Render video background:
    QCAR::State state = QCAR::Renderer::getInstance().begin();
    
#ifdef USE_OPENGL_ES_1_1
    // Set GL11 flags:
    glEnableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_NORMAL_ARRAY);
    glDisableClientState(GL_TEXTURE_COORD_ARRAY);
    glDisable(GL_LIGHTING);
#endif
    
    glDisable(GL_TEXTURE_2D);
    glEnable(GL_DEPTH_TEST);
    glEnable(GL_CULL_FACE);
    
    QCAR::Matrix44F mainModelViewMatrix;
    QCAR::Vec3F targetCenters[2]; // make this big enough to hold all your targets
    
    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
    {
        // Get the trackable:
        const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
        QCAR::Matrix44F modelViewMatrix =
        QCAR::Tool::convertPose2GLMatrix(trackable->getPose());        
        
        if (tIdx == 0)
        {
            // Make the first visible target our world center (0, 0, 0)
            // Store its modelViewMatrix and continue looking for other targets
            mainModelViewMatrix = modelViewMatrix;
            targetCenters[0].data[0] = 0.0f;
            targetCenters[0].data[1] = 0.0f;
            targetCenters[0].data[2] = 0.0f;
        }
        else
        {
            // This is another visible target
            // Find its center point in relation to the first target
            // To do this we use the matrix inverse function (SampleMath.h from the Dominoes project)
            QCAR::Matrix44F mainModelViewInverse = SampleMath::Matrix44FInverse(mainModelViewMatrix);
            QCAR::Matrix44F modelViewTranspose = SampleMath::Matrix44FTranspose(modelViewMatrix); // let's work with row-major matrices
            QCAR::Matrix44F offsetMatrix = QCAR::Tool::multiply(mainModelViewInverse, modelViewTranspose);
            
            // Transform a point on the second target by this offset matrix
            // (0, 0, 0) is the local center of the target
            QCAR::Vec4F position(0.0f, 0.0f, 0.0f, 1.0f);
            position = SampleMath::Vec4FTransform(position, offsetMatrix);
            
            // Add this position to our array
            targetCenters[1].data[0] = position.data[0];
            targetCenters[1].data[1] = position.data[1];
            targetCenters[1].data[2] = position.data[2];
        }
    }
    
    if (state.getNumActiveTrackables() > 1)
    {
#ifdef USE_OPENGL_ES_1_1
        // Load projection matrix:
        glMatrixMode(GL_PROJECTION);
        glLoadMatrixf(projectionMatrix.data);
        
        // Load model view matrix:
        glMatrixMode(GL_MODELVIEW);
        glLoadMatrixf(mainModelViewMatrix.data);
        
        // Set the color to red:
        glColor4f(1.0f, 0.0f, 0.0f, 1.0f);
        
        // Draw object:
        glVertexPointer(3, GL_FLOAT, 0, (const GLvoid*) &targetCenters[0].data[0]);
        glDrawArrays(GL_LINES, 0, 2);
#else
        
        QCAR::Matrix44F modelViewProjection;
        
        SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                                    &mainModelViewMatrix.data[0],
                                    &modelViewProjection.data[0]);
        
        glUseProgram(shaderProgramID);
        
        glVertexAttribPointer(vertexHandle, 3, GL_FLOAT, GL_FALSE, 0,
                              (const GLvoid*) &targetCenters[0].data[0]);
        
        glEnableVertexAttribArray(vertexHandle);
        
        glUniformMatrix4fv(mvpMatrixHandle, 1, GL_FALSE,
                           (GLfloat*) &modelViewProjection.data[0]);
        glDrawArrays(GL_LINES, 0, 2);
#endif
    }
    
    glDisable(GL_DEPTH_TEST);
    
#ifdef USE_OPENGL_ES_1_1        
    glDisable(GL_TEXTURE_2D);
    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_NORMAL_ARRAY);
    glDisableClientState(GL_TEXTURE_COORD_ARRAY);
#else
    glEnable(GL_TEXTURE_2D);
    glDisableVertexAttribArray(vertexHandle);
    glDisableVertexAttribArray(normalHandle);
    glDisableVertexAttribArray(textureCoordHandle);
#endif
    
    QCAR::Renderer::getInstance().end();
}

Also, I suggest using OpenGL ES 1.1 for this, as it's easier to set the line color (you don't have to mess with shaders). In the Android.mk file you can set USE_OPENGL_ES_1_1 to true.

- Kim

Update on what I've gleaned from spending more time reading:
The place where I want to be for rendering is
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame
within the else for OpenGL ES 2.0

Also to get my origin I need to grab the floats out of the Vec2f which ImageTarget's getSize returns as. Then I should be able to simply width/2, height/2.

That's all great. Where I'm still lost is all the OpenGL code. I've switched over from the teapot to some code that ksiva gave a person for displaying a 2d image.
http://ar.qualcomm.at/node/2000357

My end goal that I want to be able to do is simply do a glDrawArrays with a LINE_STRIP. I'll give the C++ code I have currently from Kim, and the Java code I have from another Android project to show where I'm wanting to get to.

C++ from Kim

        QCAR::Vec2F targetSize = ((QCAR::ImageTarget *) trackable)->getSize();

        QCAR::Matrix44F modelViewProjection;

        SampleUtils::translatePoseMatrix(0.0f, 0.0f, kObjectScale,
                                         &modelViewMatrix.data[0]);
        SampleUtils::scalePoseMatrix(targetSize.data[0], targetSize.data[1], 1.0f,
                                     &modelViewMatrix.data[0]);
        SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                                    &modelViewMatrix.data[0] ,
                                    &modelViewProjection.data[0]);

        glUseProgram(shaderProgramID);
         
        glVertexAttribPointer(vertexHandle, 3, GL_FLOAT, GL_FALSE, 0,
                              (const GLvoid*) &planeVertices[0]);
        glVertexAttribPointer(normalHandle, 3, GL_FLOAT, GL_FALSE, 0,
                              (const GLvoid*) &planeNormals[0]);
        glVertexAttribPointer(textureCoordHandle, 2, GL_FLOAT, GL_FALSE, 0,
                              (const GLvoid*) &planeTexcoords[0]);

        glEnableVertexAttribArray(vertexHandle);
        glEnableVertexAttribArray(normalHandle);
        glEnableVertexAttribArray(textureCoordHandle);

        glBindTexture(GL_TEXTURE_2D, thisTexture->mTextureID);
        glUniformMatrix4fv(mvpMatrixHandle, 1, GL_FALSE,
                           (GLfloat*)&modelViewProjection.data[0] );
        glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT,
                       (const GLvoid*) &planeIndices[0]);

Java from me

                gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);// OpenGL docs.
                // vertexBuffer was given an array of floats
		gl.glVertexPointer(2, GL10.GL_FLOAT, 0, // OpenGL docs
                                 vertexBuffer);

		gl.glDrawArrays(GL10.GL_LINE_STRIP, 0, indices.length);

		gl.glDisableClientState(GL10.GL_VERTEX_ARRAY); // OpenGL docs
		gl.glDisable(GL10.GL_CULL_FACE); // OpenGL docs

Also if someone could explain to me what planeVertices[] planeTexcoords[] planeNormals[] planeIndices[] do in her example in the link, that would also be very helpful. I believe I understand vertices and indices pretty well but the other two I don't know.