I also found this post, but my game engine doesn't have the API calls needed to set the up and right vectors for the camera. I was able to get the 3x3 matrix out of the inversed pose matrix and create a quaternion out of that, figuring it had rotation information, but setting the camera's roientation to that quaternion didn't work as expected either.

Which game engine are you using?

I'm using gamekit, but to set the camera's custom view and projection matrices, I am calling down to Ogre.

Have you seen this project? - https://github.com/airgames/vuforia-gamekit-integration

Yeah, I used that project to figure out how to use the pose and projection matrices. The original source doesn't work correctly, though. They set the custom view matrix for the camera too, but they do some weird math on it that I don't understand, and the virtual world didn't render correctly when moving the camera around.

The render function is where the camera in gamekit is set, and here's my version with the relevant bits:

	//ogre is row major, gl and vuforia(the matrices mvm and projmatrix) are column major order
	Ogre::Matrix4 modelViewMatrix = Ogre::Matrix4(
			mvm[0], mvm[4], mvm[8],  mvm[12],
			mvm[1], mvm[5], mvm[9],  mvm[13],
			mvm[2], mvm[6], mvm[10], mvm[14],
			mvm[3], mvm[7], mvm[11], mvm[15]
	);

	Ogre::Matrix4 projectionMatrix = Ogre::Matrix4(
			projmatrix[0], projmatrix[4], projmatrix[8],  projmatrix[12],
			projmatrix[1], projmatrix[5], projmatrix[9],  projmatrix[13],
			projmatrix[2], projmatrix[6], projmatrix[10], projmatrix[14],
			projmatrix[3], projmatrix[7], projmatrix[11], projmatrix[15]
	);

	Ogre::Camera *c = gkEngine::getSingleton().getCamera();

	c->setProjectionType(Ogre::PT_PERSPECTIVE);

        //works as expected
	c->setCustomProjectionMatrix(true, projectionMatrix);
	c->setCustomViewMatrix(true, modelViewMatrix);

        //can't figure out how to use the inverse of the pose matrix
	//Ogre::Matrix4 inverseModelViewMatrix = modelViewMatrix.inverse();
	//Ogre::Vector3 camTrans = inverseModelViewMatrix.getTrans();
	//Ogre::Matrix3 inversemvm3 = Ogre::Matrix3();
	//inverseModelViewMatrix.extract3x3Matrix(inversemvm3);
        //
	//Ogre::Vector3 camRight = inversemvm3.GetColumn(0); //can't use it
	//Ogre::Vector3 camUp = inversemvm3.GetColumn(1); //can't use it
	//Ogre::Vector3 camDir = inversemvm3.GetColumn(2);
	//
	//c->setPosition(camTrans);
        //
        //Option a: (according to ksiva's post)
	//c->lookAt(camDir);
        //
        //Option b:
        //Ogre::Quaternion camQuat = inverseModelViewMatrix.extractQuaternion();
        //c->setOrientation(camQuat);

I noticed that the values for y seem to be reversed when using the method that gets the translation for the camera from the inverse of the model view matrix. When I move the physical camera up the y axis, the y values are getting lower. The x and z values seem to follow what I'd expect for the virtual world. What could this mean?

See: https://developer.vuforia.com/resources/dev-guide/projection-matrix-explained

Based on the intrinsic camera calibration of the specific device (coming from the device profile) we create a GL projection matrix with the following coordinate frame:  

The camera sits at <0,0,0> and points into the positive z-direction; x-direction is to the right and y-direction is down-wards. Hence, we have a right handed coordinate system. This is not how OpenGL does it per default, but since it is just matrix math, we can do whatever we want here.

The reason why we use this coordinate frame is, because it is the same as for targets - just rotated by 90 degrees around the x-axis (for targets x-direction is to the right, y-direction is upwards in the target plane and z-direction points out of the target-plane).

There are several other articles in the KB that could be helpful - https://developer.vuforia.com/resources/dev-guide/knowledge-base-articles

Hi, I am having a similar problem. Did you manage to solve this problem and how?

I'm wondering maybe if I/we need to create our own projection matrix from the CameraDevice... not sure though.

Here is my related post: https://developer.vuforia.com/forum/qcar-api/vuforia-inverse-model-matrix-gives-wrong-effects-how-fix

Thanks

PS the top link in DavidBeard's answer does not seem to work :-(

Are these what you need?

https://developer.vuforia.com/library//articles/Solution/How-To-Use-the-Camera-Projection-Matrix

https://developer.vuforia.com/library//articles/Solution/Get-the-Camera-Position

Somebody please explain the derivation of the projection matrix. As @pbs123 has already said, the "projection matrix explained" link on David Beard's answer has been shaved. I *really* want to know what happens to my coordinates in eye space and what I'm multiplying them with and how it depends on the coordinate system (with the positive Y downwards and positive Z inwards and all that).

Dear ayushvora,

My solution on stackoverflow may help.
http://stackoverflow.com/questions/30123438/vuforia-inverse-model-matrix-produces-undesired-effects-how-to-fix

Good luck.

Thanks but that didn't exactly help me right now. I mean, I do plan to play with Vuforia matrices sometime but before doing that, because I'm a newbie, I just wanna ensure that the derivation/calculation of Vuforia's projection matrix *depends* on the fact that their +Y is downward and +Z inward. I say this because

a) My 3d model loads up on OpenGL on using Vuforia view matrix and Vuforia projection matrix 

b) but not if I replace the projection matrix with one generated by android.opengl.Matrix.frustumM

c) I do, however, see the model, if I use android.opengl.Matrix.setLookAtM for view matrix and android.opengl.Matrix.frustumM for projection but obviously, that's useless in AR setting - we wanna place the model wrt marker detected by Vuforia, right?

I think this is because the android.opengl.Matrix matrices are prepared with +Y upward and +Z outward but both the Vuforia view and projection matrices are prepared with +Y downward and +Z inward but that's what, I only *think* so - I wanna confirm it. I mean of course the two matrices will probably differ only in the signs of certain elements but that obviously makes a difference when used in OpenGL rendering.