Log in or register to post comments

Outline target area w/ line

June 16, 2011 - 11:09am #1

I am doing a simple exercise w/ the AR SDK. Similar how the buttons are outlined w/ a square draw on the screen in the VirtualButtons example, I would like to outline a target w/ opengl.

For example: A target is found - then around it's border draw a line.

What would be the process to do this. Right now I have the target pose:

const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
QCAR::Matrix44F modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose());

What do I do from here? I've just tried drawing w/ that matrix but it just draws weird lines:

glVertexAttribPointer(gvPositionHandle, 3, GL_FLOAT, GL_FALSE, 0, (const GLvoid*) &modelViewMatrix.data[0]);
glDrawArrays(GL_LINES, 0, 12);

How can I outline the target area w/ a line?

Outline target area w/ line

October 19, 2016 - 4:52am #13

My question is in the scope of the Unity extension. I'm on Unity 5.4.

How can I get the array of points in the camera frame coordinates that bounds the tracked image target? With that, I could draw the outline of the target on the video background texture.

Re: Outline target area w/ line

June 21, 2011 - 7:43pm #12

http://www.opengl.org/resources/faq/technical/transformations.htm

The vertex shader multiplies each vertex in our model by the modelviewProjection matrix. The rendering pipeline then takes each transformed triangle in our model (assuming we're rendering triangles) and determines which screen pixels to paint. The fragment shader determines the color. These two shaders together make a shader program. Open CubeShaders.h in the samples to see what these shaders look like.

glVertexAttribPointer() assigns an array of model data to a variable in our vertex shader. The address of this variable was discovered using the glGetAttribLocation() method. glEnableVertexAttribArray() simply enables this array, ensuring it will be sent to the shader when glDrawElements() is called. glUniformMatrix4fv() sends our modelviewProjection matrix to the vertex shader. All of these functions are just ways of mapping data we have in C++ to variables in the shader program.

The OpenGL ES 2.0 API is helpful: http://www.khronos.org/opengles/sdk/docs/man/

Sorry, that was a book. Rendering is fun!

- Kim

Re: Outline target area w/ line

June 21, 2011 - 2:36pm #11

Thanks.

A couple more clarifications:

The modelViewMatrix is the 3d pose which is a marix which represents where the target is relative from the viewer, right?

What is the modelViewProjection matrix?

What is the modelViewProjection matrix?

What does multiplying the projection matrix by the model view matrix do?
SampleUtils::multiplyMatrix(&projectionMatrix.data[0], &modelViewMatrix.data[0], &modelViewProjection.data[0]);

The shader scripts rotate, scale and move the object right?
glUseProgram(vbShaderProgramID);

This simply sets the data to draw right?
glVertexAttribPointer(vbVertexHandle, 3, GL_FLOAT, GL_FALSE, 0, (const GLvoid*) &vbVertices[0]);

What does this do?
glEnableVertexAttribArray(vbVertexHandle);

Also I've tried to find documentation but I dont under stand this gl function either:
glUniformMatrix4fv(mvpMatrixHandle, 1, GL_FALSE, (GLfloat*)&modelViewProjection.data[0] );

Re: Outline target area w/ line

June 20, 2011 - 6:58pm #10

The shader language is GLSL, which is typical for OpenGL ES applications. The shader programs run on the GPU, and with OpenGL ES 2.0 you must use a shader program to render anything (it uses a programmable pipeline rather than a fixed function pipeline).

For more info: http://www.khronos.org/opengles/

- Kim

Re: Outline target area w/ line

June 20, 2011 - 10:22am #9

Thanks for the clarification.

I'll have to study up more on shaders, but from what I understand it's a script that opengl can to the gpu instead of the cpu right? For faster performance...

In what language are those shaders written in the sample projects?

Re: Outline target area w/ line

June 19, 2011 - 8:12pm #8
So it's not possible to rotate/scale/move the objects w/o using shaders? I can't just use matrix transforms?

If you were using OpenGL ES 1.1 you could apply transforms directly to the matrix stack. For OpenGL ES 2.0 you must use shaders, however, and that means feeding your projection matrix, modelview matrix, and extra transforms to the shader. Typically you premultiply all these matrices together before passing them to the shader, that's the modelViewProjection in the sample code. The shader then transforms each vertex in your array using this matrix.

From the vertex shader:

gl_Position = modelViewProjectionMatrix * vertexPosition;

- Kim

Re: Outline target area w/ line

June 17, 2011 - 7:07pm #7

Yes opengl 2.0. Hmm, I agree. I copied my render code to the VirtualButtons project and it worked. When I comment glUseProgram on my project it draws a triangle on the plane of the screen. However, if uncomment glUseProgram I see nothing drawn on the screen. There must be something else wrong w/ my project -- maybe the shader program code isn't getting loaded correctly or something. I'll look into it.

Side question though. So it's not possible to rotate/scale/move the objects w/o using shaders? I can't just use matrix transforms?

Re: Outline target area w/ line

June 17, 2011 - 6:35pm #6

Are you trying to use OpenGL ES 1.x? Shaders are required for OpenGL ES 2.0, and both of your code samples are written for OpenGL ES 2.0. I just opened the VirtualButtons project, commented out all the code in the renderFrame method and replaced it with your code and my suggested changes. It worked as expected (lines drawn on the target).

This line could potentially cause you problems if you have multiple targets:

const QCAR::Trackable* trackable = state.getActiveTrackable(0);

That should be:

const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);

MultiTargets does use a shader, look for this line:

glUseProgram(shaderProgramID);

It renders a textured cube rather than lines, however.

- Kim

Re: Outline target area w/ line

June 17, 2011 - 5:43pm #5

Hi, thanks for your support. I tried what you suggested and for simplicity, I'm just drawing a triangle now. It draws a static triangle on the screen (i.e. it just stays flat on screen plane).

If I uncomment the glUseProgram line for some reason it doesn't draw the triangle anymore. Do I have to use this shader program? Multitargets example draws borders around the targets w/0 using gluseprogram. I'll admit ignorance, but it seems that the shader programs lineMeshVertexShader and lineFragmentShader are just for color? Correct?

My other question is, (assuming the line shaders don't effect rotation) why doesn't my triangle move w/ the target? I have all the matrix transform code just like the examples: multiplyMatrix and glUniformMatrix4fv.

So basically my problem is my triangle isn't moving like an AR object should.

{
    // Clear color and depth buffer 
     glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

     // Render video background:
     QCAR::State state = QCAR::Renderer::getInstance().begin();

     glEnable(GL_DEPTH_TEST);
     glEnable(GL_CULL_FACE);
    
      for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
      {
        
        //get target object 
         const QCAR::Trackable* trackable = state.getActiveTrackable(0);
         
         //get qcar pose 
         QCAR::Matrix44F modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
         
         //get target dimensions
         const QCAR::ImageTarget* target =   static_cast<const QCAR::ImageTarget*>(trackable);
         QCAR::Vec2F target_size = target->getSize();
         GLfloat x= target_size.data[0]/2;
         GLfloat y = target_size.data[1]/2;         

        GLfloat vbVertices[24];
        vbVertices[0]=0.5f;
        vbVertices[1]=-0.5f;
        vbVertices[2]=0.0f;
        
        vbVertices[3]=-0.5f;
        vbVertices[4]=0.5f;
        vbVertices[5]=0.0f;
        
        vbVertices[6]=-0.5f;
        vbVertices[7]=0.5f;
        vbVertices[8]=0.0f;
        
        vbVertices[9]=-0.5f;
        vbVertices[10]=-0.5f;
        vbVertices[11]=0.0f;
        
        vbVertices[12]=-0.5f;
        vbVertices[13]=-0.5f;
        vbVertices[14]=0.0f;
        
        vbVertices[15]=0.5f;
        vbVertices[16]=-0.5f;
        vbVertices[17]=0.0f;
        
         
         //matrix transformations
         QCAR::Matrix44F modelViewProjection;
         SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                                     &modelViewMatrix.data[0],
                                     &modelViewProjection.data[0]);
         
        LOG("sharder program id is ************* : %d", vbShaderProgramID);
         //glUseProgram(vbShaderProgramID);
        
        //set data                             
        glVertexAttribPointer(vbVertexHandle, 3, GL_FLOAT, GL_FALSE, 0,  (const GLvoid*) &vbVertices[0]);
        
        //apparently you need this to drqw
        glEnableVertexAttribArray(vbVertexHandle);
            
        glUniformMatrix4fv(mvpMatrixHandle, 1, GL_FALSE, (GLfloat*)&modelViewProjection.data[0] );

        //go!
        glDrawArrays(GL_LINES, 0, 6);
        SampleUtils::checkGlError("OkapiLove renderFrame");
          
        glDisableVertexAttribArray(vbVertexHandle);
          
      }
         
    QCAR::Renderer::getInstance().end();
}

Re: Outline target area w/ line

June 17, 2011 - 4:04pm #4

So close! You're forgetting to bind a shader program. If you're working with the VirtualButtons sample you probably want this:

glUseProgram(vbShaderProgramID);

Then comment the call to glUniformMatrix4fv back in. You'll see that your lines are a bit off, but I think you can take it from there :)

- Kim

Re: Outline target area w/ line

June 17, 2011 - 3:47pm #3
ksiva wrote:

Take a look at the vbVertices array in the VirtualButtons sample to understand how to set up the corner vertices for line rendering.

HI, this is my code to render a border around the image target per your suggestions. It doesn't work yet and I was hoping for corrections. It just draws a random line across the screen that doesn't move as you move the handset.

     // Clear color and depth buffer 
     glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

     // Render video background:
     QCAR::State state = QCAR::Renderer::getInstance().begin();

     glEnable(GL_DEPTH_TEST);
     glEnable(GL_CULL_FACE);
    
      for(int tIdx = 0; tIdx < state.getNumActiveTrackables(); tIdx++)
      {
        
        //get target object 
         const QCAR::Trackable* trackable = state.getActiveTrackable(0);
         
         //get qcar pose 
         QCAR::Matrix44F modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(trackable->getPose());
         
         //get target dimensions
         const QCAR::ImageTarget* target =   static_cast<const QCAR::ImageTarget*>(trackable);
         QCAR::Vec2F target_size = target->getSize();
         GLfloat x= target_size.data[0]/2;
         GLfloat y = target_size.data[1]/2;         

        GLfloat vbVertices[24];
        vbVertices[0]=-x;
        vbVertices[1]=y;
        vbVertices[2]=0.0f;
        vbVertices[3]=x;
        vbVertices[4]=-y;
        vbVertices[5]=0.0f;
        vbVertices[6]=x;
        vbVertices[7]=y;
        vbVertices[8]=0.0f;
        vbVertices[9]=x;
        vbVertices[10]=-y;
        vbVertices[11]=0.0f;
        vbVertices[12]=x;
        vbVertices[13]=-y;
        vbVertices[14]=0.0f;
        vbVertices[15]=-x;
        vbVertices[16]=-y;
        vbVertices[17]=0.0f;
        vbVertices[18]=-x;
        vbVertices[19]=-y;
        vbVertices[20]=0.0f;
        vbVertices[21]=-x;
        vbVertices[22]=y;
        vbVertices[23]=0.0f;
        
         
         //matrix transformations
         QCAR::Matrix44F modelViewProjection;
         SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                                     &modelViewMatrix.data[0],
                                     &modelViewProjection.data[0]);
         
        //set data                             
        glVertexAttribPointer(vbVertexHandle, 3, GL_FLOAT, GL_FALSE, 0,  (const GLvoid*) &vbVertices[0]);
        
        //apparently you need this to drqw
        glEnableVertexAttribArray(vbVertexHandle);
            
        //glUniformMatrix4fv(mvpMatrixHandle, 1, GL_FALSE, (GLfloat*)&modelViewProjection.data[0] );

        //go!
        glDrawArrays(GL_LINES, 0, 8);
          
        glDisableVertexAttribArray(vbVertexHandle);
          
      }
         
    QCAR::Renderer::getInstance().end();

What do you think the problem is?

Re: Outline target area w/ line

June 17, 2011 - 7:43am #2

The modelview matrix is used for positioning the target in relation to the camera. It is not an array of vertices for the corners of the target, which is what you want.

Using that modelview matrix, you can assume that the center of the target is at the world origin (0, 0, 0). So you just need to know the X/Y offset of the four corners. For this, use the half width and height of the target. You can find this with the ImageTarget::getSize method (you'll need to cast your Trackable to an ImageTarget first).

Take a look at the vbVertices array in the VirtualButtons sample to understand how to set up the corner vertices for line rendering.

- Kim

Log in or register to post comments