Log in or register to post comments

Front Camera Issue (mirror image)

April 30, 2015 - 1:58am #1

Vuforia SDK Version: 4.0 -

Description with steps to reproduce:

Have an app built with Vuforia 4.0 + Unity, where there is an original implementation of user-defined surface(markerless augmentation) feature, which can be enabled/disabled with a button in the app.

Also,  in the app the user can switch bet. front and rear camera.

Encountered an issue, when using the markerless feature with front camera.

 

The issue is like:

[Use Front Camera]

  a. Markerless --- Not working properly (expect to see the CG character's face but instead see its  back)

  b. Normal target tracking --- works fine

[Use Rear Camera]

  a. Markerless --- works fine

  b. Normal target tracking --- works fine

 

Do you happen to have an idea on what could be wrong? Or, Can you let me knowwhat kind of info you need for further investigation. 

 

Thank you!

-

Development OS (Mac OS X, Windows, Linux): NA -

Mobile OS and Version: Android and/or iOS -

Mobile Device Manufacturer and Model name: NA -

Do the Vuforia Sample Applications show the same behavior?:

Front Camera Issue (mirror image)

June 11, 2015 - 9:30pm #15

Glad to know that it worked. Thanks for the update.

Front Camera Issue (mirror image)

June 11, 2015 - 5:54pm #14

Hi,

 

Thank you very much for your code.

Finally, the camera is fixed completely!

 

So, overall, I have done the followings,

1. Attach post #3 code to my Camera which is rendering only the model

-> To fix model's rendering problem

2. Attach post #13 code to Vuforia's "Camera" under "ARCamera"

-> To fix the camera flipping problem on Nuxs 6 caused by the fix 1

 

Again, thank you for searching the solution.

Front Camera Issue (mirror image)

June 10, 2015 - 8:58pm #13

Here's a script that you can try with the ImageTargets sample app on your Nexus 6. Attach it to the Camera object which is parented to the ARCamera. I was able to switch between the front and back cameras successfully. If it works with the sample, then try it also in your project.

using UnityEngine;
using System.Collections;

using Vuforia;

[RequireComponent(typeof(Camera))]
public class Nexus6FrontCamFix : MonoBehaviour
{

	private bool mFlipped = false;
	private bool mFlipVertical = true;
	private bool mFlipHorizontal = true;
	private bool mFirstCall = true;

	private CameraDevice.CameraDirection direction;
	private Transform backgroundPlane;
	private Vector3 planeRearCam;
	private Vector3 planeFrontCam;

	private bool firstRun = true;

	void Start()
	{
		backgroundPlane = transform.GetComponentInChildren<Transform>();
		float x = backgroundPlane.transform.localScale.x;
		float y = backgroundPlane.transform.localScale.y;
		float z = backgroundPlane.transform.localScale.z;
		
		planeRearCam = new Vector3(x, y, z);
		planeFrontCam = new Vector3(x * -1, y * -1, z);
	}

	void OnPreCull()
	{
		if (CameraDevice.Instance != null) {
			direction = CameraDevice.Instance.GetCameraDirection();
			switch (direction) {
				case CameraDevice.CameraDirection.CAMERA_FRONT:
					if (firstRun) {
						backgroundPlane.localScale = planeFrontCam;
						firstRun = false;
					}
					break;
				
				case CameraDevice.CameraDirection.CAMERA_BACK:
					backgroundPlane.localScale = planeRearCam;
					mFlipped = false;
					mFirstCall = true;
					break;
			}
		} else {
			Debug.Log("<color=cyan>CameraDevice.Instance = null</color>");
			return;
		}

		if (!mFlipped && direction == CameraDevice.CameraDirection.CAMERA_FRONT) {
			if (!mFirstCall) {  // don't flip on first call to OnPreCull()

				Camera cam = this.GetComponent<Camera>();
				Vector3 flipScale = new Vector3(mFlipHorizontal ? -1 : 1, mFlipVertical ? -1 : 1, 1);
				Matrix4x4 projMat = cam.projectionMatrix * Matrix4x4.Scale(flipScale);
				cam.projectionMatrix = projMat;
				mFlipped = true;
			}
			mFirstCall = false;
		}
	}
	
	void OnPreRender()
	{
		if (direction == CameraDevice.CameraDirection.CAMERA_FRONT) {
			if ((mFlipVertical && !mFlipHorizontal) || 
				(mFlipHorizontal && !mFlipVertical)) {
				GL.SetRevertBackfacing(true);
			}
		}
	}
	
	// Set it to false again because we don't want to affect all other cameras.
	void OnPostRender()
	{
		if (direction == CameraDevice.CameraDirection.CAMERA_FRONT) {
			if ((mFlipVertical && !mFlipHorizontal) || 
				(mFlipHorizontal && !mFlipVertical)) {
				GL.SetRevertBackfacing(false);
			}
		}
	}
}

 

Front Camera Issue (mirror image)

June 3, 2015 - 6:40pm #12

In all 3 of the screenshots the teapot is in same spot on screen and doesn't appear to be parented to the target. Also, when both flip dimensions are the same value, the GL.SetRevertBackfacing() is not run, but in all three screenshots the teapot appears to have backfacing on. I'd suggest parenting the teapot to the target, so that it is only displayed when the target is being tracked and verifying that your code is following example from workaround in other thread. Also, tracking the target and augmentation will provide clue as to flipping when moving the camera in relation to target.
 

Front Camera Issue (mirror image)

June 3, 2015 - 6:15pm #11

Hi,

Here are the screenshots of different values.

Thank you.

AttachmentSize
Image icon false-true.png1.34 MB
Image icon true-true.png1.33 MB
Image icon false-false.png1.35 MB

Front Camera Issue (mirror image)

June 1, 2015 - 8:05pm #10

Thank you for the screenshots. The watermark makes it easy to see that ref1 is flipped vertically and ref2 is flipped horizontally. Assuming that you are using the code provided in the other thread, try the following 3 combinations to see if one works:

private bool mFlipVertical = false;
private bool mFlipHorizontal = true;
private bool mFlipVertical = true;
private bool mFlipHorizontal = true;
private bool mFlipVertical = false;
private bool mFlipHorizontal = false;

 

Front Camera Issue (mirror image)

June 1, 2015 - 4:51am #9

Hi,

Sorry for the late response.

Here is the result of what I tried.

 

>see if you can get front cam fixed in our ImageTargets sample using the steps in the Nexus 6 thread

With the ImageTargets sample, the camera vertically flips for the starting rear camera. After I change to front camera, it is same as before (vertically flipped. please see ref1). And when I change back to rear camera again, it is normally displayed (no flipping).

 

>It would help to see how the background is flipped in a screenshot.

Thank you for your explanation.

I have attached the screenshot of the front camera with the same code as the one I tried with ImageTarget sample (ref2).

(the boolean values are as you stated in the last post )

 

Thank you.

 

AttachmentSize
Image icon ref1.png1.57 MB
Image icon ref2.png1.43 MB

Front Camera Issue (mirror image)

May 27, 2015 - 9:28pm #8

Yes, see if you can get front cam fixed in our ImageTargets sample using the steps in the Nexus 6 thread:

https://developer.vuforia.com/forum/issues-and-bugs/front-camera-upsidedown-nexus6

It would help to see how the background is flipped in a screenshot. The Vuforia watermark in the bottom left can be helpful indicator to see how your background is being drawn. These are the two booleans that can change the flip behavior in the workaround script:

private bool mFlipVertical = true;
private bool mFlipHorizontal = false;

 

Front Camera Issue (mirror image)

May 26, 2015 - 9:33pm #7

Thank you for your reply.

 

>You can experiment with the initial boolean values to see how the background and matrix is flipped.

I have tried like "if (cam.projectionMatrix != projMat){ //set matrix code }" but no success.

 

>If you can provide a screenshot of what it looks like that might help.

Which screenshot might help you to get this going?

 

>Also, to isolate the issue, could you apply our workaround to our ImageTargets sample app on the Nexus 6 and see if it works?

Yes, it works, but only on the first rear side, not the front side.

(probablly because I have to reset the flag "mFirstCall" again?)

 

Front Camera Issue (mirror image)

May 26, 2015 - 7:32pm #6

You can experiment with the initial boolean values to see how the background and matrix is flipped. If you can provide a screenshot of what it looks like that might help. Also, to isolate the issue, could you apply our workaround (not using 30-40 count) to our ImageTargets sample app on the Nexus 6 and see if it works?

Front Camera Issue (mirror image)

May 25, 2015 - 3:22am #5

Hi,

 

Thank you very much for your information.

 

I have tired your fix, but the problem was not solved completely.

What I mean by "completely" is that the thread says I have to  skip the first OnPreCull() call (thread number 4 of the given url), but I actually have to wait around 30 to 40 calls.

 

I am guessing the Camera.projectionMatrix is overridden by Vuforia somewhere in the code, but I could not figure out exactly when that is.

 

Should I still use the estimated wait calls for this, or are there any flags that Vuforia owns, so I can check the value to check for the timing?

It is definitely possible that I am wrong, so please let me know if there are better ways.

 

For the last, I paste my code for your reference.

public class CameraRotate : MonoBehaviour {

	private int count = 0;
	private bool mFlipReady = false;
	private bool mFlipped = false;
	private bool mFlipVertical = true;
	private bool mFlipHorizontal = false;
	
	void OnPreCull () {
		if (!mFlipped) {
			if(mFlipReady)
			{
				Camera cam = GetComponent<Camera>();
				Vector3 flipScale = new Vector3 (mFlipHorizontal ? -1 : 1, mFlipVertical ? -1 : 1, 1);
				Matrix4x4 projMat = cam.projectionMatrix * Matrix4x4.Scale(flipScale);
				cam.projectionMatrix = projMat;
				mFlipped = true;
			}

			if(count >= 40){
				mFlipReady = true;
			}else{
				count++;
			}
		}
	}
	
	void OnPreRender () {
		if ((mFlipVertical && !mFlipHorizontal) || 
		    (mFlipHorizontal && !mFlipVertical)) {
			GL.SetRevertBackfacing (true);
		}
	}
	
	// Set it to false again because we don't want to affect all other cameras.
	void OnPostRender () {
		if ((mFlipVertical && !mFlipHorizontal) || 
		    (mFlipHorizontal && !mFlipVertical)) {
			GL.SetRevertBackfacing (false);
		}
	}

}

 

 

Front Camera Issue (mirror image)

May 21, 2015 - 7:43pm #4

Front Camera Issue (mirror image)

May 20, 2015 - 7:53pm #3

Tried the following quick fix (also mentioned in another post) but still observing some issues on a specific device.

1. The Fix

Also mentioned in: https://developer.vuforia.com/forum/general-discussion/problemswap-camerafront-cameraback-0

Currently using an additional camera for rendering the content.  Applied the following fix.

 
 
using UnityEngine;
 
public class BackfaceCulling : MonoBehaviour
 
{
 
  void OnPreRender() {
 
    GL.SetRevertBackfacing(false);
 
  }
 
}

2. Results, issue

Tested on Nexus 5 (Android 5.1) and confirmed it is fixed. thank you!

However, when testing on Next 6 (Android 5.1) , the AR character is OK but the camera view (background) is upside down.

Is this related to the mirror video background setting of ARCamera (=DEFAULT) ?

Also, thought both devices are on Android 5.1, we are not yet able to tell whether this is OS dependent or HW dependent issue.

Can you please check our sample (shared with you separately) and kindly share with us your ideas? Thank you!

 

 

Front Camera Issue (mirror image)

April 30, 2015 - 11:36am #2

I'm not noticing any problem using front camera with UserDefinedTargets (UDT) 4.0.105 sample app. The top of the teapot appears with both the front and back cameras as expected. If you place your geometry next to the teapot in the UDT 4.0.105 sample, does it display correctly?

 

Log in or register to post comments