Log in or register to post comments

How to set the modelViewMatrix

April 2, 2013 - 4:40am #1

Hi, 

i want to change the tracking part of Image Target sample.The tracking will be based on detection :every frame i detect the face then i put my 3d model.So i'm not using any trackable as it's done in the original sample of the ImageTarget.My problem now is to show the 3D model. i know that i have to set the ModelViewMatrix but i don't know how can i change the result->getPose()getpose()? how can i show the 3D model related to the coordinate of the detected face?is there any  function to convert detected point of the face and used it to show the 3d model?

QCAR::Matrix44F modelViewMatrix =
            QCAR::Tool::convertPose2GLMatrix(result->getPose());

 

How to set the modelViewMatrix

April 12, 2013 - 12:33am #19

Hi, in computer graphics we usually talk about ModelView matrix for the 3D transformation part (rotation, translation and scaling), and "Projection" matrix for the projective transformation (for instance the camera perspective).

The calibration parameters of the camera are used for building the Projection matrix.

We have some tutorials about these concepts here:

https://developer.vuforia.com/resources/dev-guide/projection-matrix-explained

https://developer.vuforia.com/resources/dev-guide/pose-matrix-explained

https://developer.vuforia.com/resources/dev-guide/positioning-3d-content

 

How to set the modelViewMatrix

April 11, 2013 - 3:13pm #18

i have found a solution for the facePoseZ but my model doesn't rotate it seams like the faceposeX and the face poseY are false in my case so can i get the rotation and translation matrix extrinsic parameters of the calibration).i know that to project a P(X,Y,Z) into Q(x,y) we pass par the calibration: Q=I.P 
so i wanna ask this question and maybe this not the right place to ask it: does the I matrix represents extrinsic and intrinsic parameters of the calibration?
thank you in advance

How to set the modelViewMatrix

April 9, 2013 - 7:37am #17

Keep in mind that the Teapot mesh in the ImageTargets sample is defined so to look fine when displayed on a 20 - 30 cm targets (with size expressed in millimeters) using an object scale factor of 3 (see kObjectScale variable).

So, you may need to adjust the scale of the model as well, or somehow taking into account the unity of measure of the dimensions returned by your face detection system.

 

How to set the modelViewMatrix

April 9, 2013 - 4:33am #16

i done it but the teapot became too big  just when my face is in the camera direction (i translate my face in the camera direction the teapot becomes  away from my face and big)else there  is  no change or may be it is not remarquable.may be i still have to guess the value.

How to set the modelViewMatrix

April 8, 2013 - 4:06am #15

Hi, the problem with 3D tracking is that you really need to have the depth information ("faceZ"), otherwise it is not a 3D problem, it becomes 2D, but then you loose your ability to work and reason with 3D objects.

One way to go could be by estimating (or trying to guess / estimate) the depth of the face (i.e. its distance from the camera) using some other information; for example, does your face detection software provides you with some information like the size of the face or the distance between the eyes, or something else that could be used as an estimation of the face size ?

if yes, then you could estimate the face distance (roughly at least) based on the face size... this approach even if not extremly accurate, could probably be an acceptable way of getting it to work in 3D...

 

How to set the modelViewMatrix

April 8, 2013 - 4:00am #14

hi again,

i tried to find the poseZ but  with no result. so i wounder can i integrate a 2d image in the video frame and use it as target to make my 3d object  track this target?is it possible? really i don't know how can i make my 3d model track my face.

How to set the modelViewMatrix

April 3, 2013 - 10:03am #13

Good.  Let me know how it goes with the facePosZ....

How to set the modelViewMatrix

April 3, 2013 - 9:48am #12

thank you.i forget that i gave to the faceposeX and faceposeY a constant value.now i have just to search how to get the faceposeZ value.

 

How to set the modelViewMatrix

April 3, 2013 - 8:41am #11

Yes, but have you done the extra steps that I highlight in the code:

 

SampleUtils::translatePoseMatrix(facePosX, facePosY, facePosZ,
                                                 &modelViewMatrix.data[0]);

How to set the modelViewMatrix

April 3, 2013 - 7:48am #10

i have done it already :

  for (int i = 0; i < 16; ++i)
                 modelViewMatrix.data[i] = 0;
                 modelViewMatrix.data[0] = 1.0f;
                 modelViewMatrix.data[5] = 1.0f;
                 modelViewMatrix.data[10] = 1.0f;
                 modelViewMatrix.data[15] = 1.0f;
 

but the 3dmodel is attached to the screen not to the face. i can't rotate around the 3d object.

How to set the modelViewMatrix

April 3, 2013 - 6:33am #9

Hi,

if the reference frame is as you describe, then it is the same as the one used by Vuforia (note: Vuforia also uses a reference frame which is like OpenGL but rotated 180 degrees around X, which makes the Z axis to "enter the screen");

then it should be easy to setup your "pose" matrix based on that;

suppose you have your face detected as a point at position (X,Y,Z), so suppose you have these three variables defined:

float facePosX = some value...;

float facePosY = some value ...;

float facePosZ = some value ....;

Then you should be able to compute your modelview matrix (like in the image targets sample, see ImageTargets.cpp), using this code:

// set the modelview matrix to identity matrix
        modelViewMatrix.data[0] = 1.0f;
        modelViewMatrix.data[1] = 0.0f;
        modelViewMatrix.data[2] = 0.0f;
        modelViewMatrix.data[3] = 0.0f;
        modelViewMatrix.data[4] = 0.0f;
        modelViewMatrix.data[5] = 1.0f;
        modelViewMatrix.data[6] = 0.0f;
        modelViewMatrix.data[7] = 0.0f;
        modelViewMatrix.data[8] = 0.0f;
        modelViewMatrix.data[9] = 0.0f;
        modelViewMatrix.data[10] = 1.0f;
        modelViewMatrix.data[11] = 0.0f;
        modelViewMatrix.data[12] = 0.0f;
        modelViewMatrix.data[13] = 0.0f;
        modelViewMatrix.data[14] = 0.0f;
        modelViewMatrix.data[15] = 1.0f;

        SampleUtils::translatePoseMatrix(facePosX, facePosY, facePosZ,
                                                 &modelViewMatrix.data[0]);

        SampleUtils::scalePoseMatrix(kObjectScale, kObjectScale, kObjectScale,
                                             &modelViewMatrix.data[0]);

        SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
                                    &modelViewMatrix.data[0] ,
                                    &modelViewProjection.data[0]);

The only issue I can think of is about the value of facePosZ (i.e. the depth of the face), which perhaps you don't have...

 

How to set the modelViewMatrix

April 3, 2013 - 6:16am #8

 

I'm using opencv. It is a right-handed system, with the world X and Y aligned with the image x and y
it have the same handednes as opengl , but the OpenCV one is rotated by pi around the x axis. This can be represented by the following rotation matrix:
 
 1  0  0
 0 -1  0
 0  0 -1
 
i'm using viola & jones algorithm for the detection.if i use the SURF point does it made differences?
 

How to set the modelViewMatrix

April 2, 2013 - 8:47am #6

how can i made my points in relation to the camera pose. What is the way out in that case?

How to set the modelViewMatrix

April 2, 2013 - 7:34am #5

then my answer will be no, it return the Matrix of rectangle independently from the position of the camera.i can just get the points of the returned rectangle.

How to set the modelViewMatrix

April 2, 2013 - 7:00am #4

I think that 'pose w/ respect to the camera' means that the position is defined in relation to the camera position, where the camera is at 0,0,0.

How to set the modelViewMatrix

April 2, 2013 - 6:55am #3

the detection return a MatofRect for face .i haven't understood what do you mean by  

does it provide a pose with respect to the camera ?

could you please explain this point ?

How to set the modelViewMatrix

April 2, 2013 - 4:48am #2

Hi,

what does you face detection provides ? does it provide a pose with respect to the camera ?

a 3D point in the camera reference frame ? (and if yes, how are the cmera axis defined ?)

 

Log in or register to post comments