Topic locked

Image data doesn't update after camera is re-initialized

October 15, 2012 - 1:56am #1

In Android it seems that the image pixel data isn't updated after the camera device is de-initialized and re-initialized in Unity. This happens when the user presses the Android menu button or by changing unity scenes.

To give you some background, I am using the image pixel data to find colors around the marker. Everything works great until the camera is de-initialized and re-initialized at which time the camera still picks up markers however the image data is no longer being updated.

I can simulate it easily by calling CameraDevice.Instance.Stop, Deinit, Init, Start ... then when I call GetCameraImage the data is never updated. It looks like it is the same image data from when the camera was stopped however the image on the screen is updating and targets are being tracked as per normal.

Is there a fix or a work around?

Image data doesn't update after camera is re-initialized

November 21, 2014 - 4:22am #12

I solved my problem only by using this:

 

>> In CameraDevice.cs I make sure the camera images are cleared when it is deinitialized by adding "mCameraImages.Clear();" to the start of the Deinit() function.

 

Other suggestions shown here did not work in my case.

Do not ask me how i got to cameradevice.cs in unity extension, not sure if it was legal :)

Actually i think it's a bug and gotta be reported.

 

I think it's still not fixed due to 99% vuforia users do not need to get camera pixel colors in runtime.

It stops providing correct data after android app returning from sleep.

Huge thanks to joshsavage, by the way. Without his post i'd never solve that.

Image data doesn't update after camera is re-initialized

November 8, 2014 - 4:07am #11

You need to reset the frame format in the OnApplicationPause;

see the solution I already posted previously and that huythach also used successfully.

 

See also:

https://developer.vuforia.com/forum/faq/unity-how-can-i-reset-frame-format-scene-reload

 

 

 

Image data doesn't update after camera is re-initialized

November 7, 2014 - 11:31am #10

Hello.

 

I have just the same issue as joshsavage originally had.

I call CameraDevice.Instance.GetCameraImage() (to get some pixel colors at marker) and i get nice images just until my android app pauses.

After resuming the app continues correctly tracking the marker and displays proper video at background, but the data returned by GetCameraImage is wrong. It contains the still picture camera was showing at the moment app fell asleep.

I use vuforia unity extension (C# scripts) so i cant seemingly get enough acces to call mCameraImages.Clear() at any point...

So - what do i do?

Image data doesn't update after camera is re-initialized

December 23, 2013 - 7:50am #9

Sure. Just a typo.

 

Image data doesn't update after camera is re-initialized

December 22, 2013 - 8:06pm #8

Thank AlessandroB,

But OnApplicationPause not OnApplicationPaused

Image data doesn't update after camera is re-initialized

December 20, 2013 - 2:38am #7

Hi, this example snippet shows how to set / reset the Frame Format, which works with Vuforia 2.8.7:

public class ImageFormat : MonoBehaviour, ITrackerEventHandler {

	private Rect mCurrentFormatRect;
	private Rect mDebugRect;
	private Rect mImageInfoRect;
	
	private string mDebugMsg = "";
	private string mImageInfo = "";
	private Image.PIXEL_FORMAT mFormat = Image.PIXEL_FORMAT.GRAYSCALE;
	
	private bool mRegisteredFormat = false;
	private bool mQCARInitialized = false;

	// Use this for initialization
	void Start () {
		QCARBehaviour qcar = (QCARBehaviour)FindObjectOfType(typeof(QCARBehaviour));
		if (qcar) {
			qcar.RegisterTrackerEventHandler(this);
		}
		else {
			Debug.LogError("Could not find QCARBehaviour (i.e. ARCamera) in the scene");
		}

		int w = Screen.width;
		int h = Screen.height;
		mCurrentFormatRect = new Rect(w/2 - 200, h/2 + 130, 350, 30);
		mImageInfoRect = new Rect(w/2 - 200, h/2 + 160, 350, 30);
		mDebugRect = new Rect(w/2 - 200, h/2 + 190, 350, 30);
	}

	// Implementing OnInitialized() 
	// of ITrackerEventHandler interface
	public void OnInitialized() {
		mQCARInitialized = true;
		mRegisteredFormat = false;
	}

	// Implementing OnTrackablesUpdated() 
	// of ITrackerEventHandler interface
	public void OnTrackablesUpdated() {
	}
        
	// This is called whenever the app gets paused
	void OnApplicationPaused(bool paused) {
		if (paused) {
			// invalidate registered format if app has been paused
			mRegisteredFormat = false;
		}
	}
	
	// Update is called once per frame
	void Update () {
		// Skip if QCAR has not been initialized yet
		if (mQCARInitialized) {
			if (!mRegisteredFormat) { 
				//first time update or first resume after pause
				//see OnApplicationPaused() above
				mFormat = Image.PIXEL_FORMAT.GRAYSCALE;// Choose the Frame Format you want here
				CameraDevice.Instance.SetFrameFormat(mFormat, false);
				if (CameraDevice.Instance.SetFrameFormat(mFormat, true)) {
					mDebugMsg = mFormat.ToString() + " successfully set.";
					mRegisteredFormat = true;
				}
				else {
					mDebugMsg = "Failed to set RGB888.";
					mFormat = Image.PIXEL_FORMAT.UNKNOWN_FORMAT;
				}
			}

			Image img = CameraDevice.Instance.GetCameraImage(mFormat);
			if (img != null) {
				mImageInfo = img.Width + " x " + img.Height + " " + "Pixels: " + img.Pixels[0] + ", " + img.Pixels[1] + ", " + img.Pixels[2] + " ...";
			}
			else {
				mImageInfo = "Can't get Image for " + mFormat.ToString();
			}
		}
	}
	
	void OnGUI() {
		GUI.skin.box.fontSize = 18;
		GUI.Box(mCurrentFormatRect, "Current Format: " + mFormat.ToString());
		GUI.Box(mImageInfoRect, "Image: " + mImageInfo);
		GUI.Box(mDebugRect, "Messages: " + mDebugMsg);
	}
}

 

Image data doesn't update after camera is re-initialized

December 19, 2013 - 4:29pm #6

Joshsavage, 

In Vuforia 2.6.7, I followed your comment and it worked. However, when I update to Vuforia 2.8.7, I could not find where is CameraDevice.cs file. Do you have any suggestion?

Image data doesn't update after camera is re-initialized

October 23, 2012 - 8:01pm #5

I only need it in one scene so I am not persisting the camera instance.

Image data doesn't update after camera is re-initialized

October 22, 2012 - 7:19pm #4

Thanks for posting that. So are you persisting the camera instance across scenes? - e.g. using DontDestroyOnLoad()

Image data doesn't update after camera is re-initialized

October 17, 2012 - 8:57pm #3

I fixed the issue. Note this issue only occurs the second time the AR scene is created e.g. on changing scenes or returning to the app. 

1) In CameraDevice.cs I make sure the camera images are cleared when it is deinitialized by adding "mCameraImages.Clear();" to the start of the Deinit() function.

2) In QCARBehaviour.cs I force the CameraDevice into RGB888 format after the camera is started. It's a bit of a hack but found that sometimes when the frame format is set in my start method it doesn't work. (perhaps the camera hasn't initialized at that stage, I haven't had time to investigate further)

// Start the camera and trackers and update the camera projection

private void StartQCAR(){

        CameraDevice.Instance.Init();

        ConfigureVideoBackground();

        CameraDevice.Instance.SelectVideoMode(CameraDeviceModeSetting);

        CameraDevice.Instance.Start()

        CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.RGB888,true);

 

Image data doesn't update after camera is re-initialized

October 17, 2012 - 10:28am #2

Can you post a sample of the code that you're using to test this?  - or PM me if you don't want to make it public.

Also which version # of the Unity extension are you using? And which device(s) are you observing this on?

Topic locked