Log in or register to post comments

Frame rate drops even after SetFrameFormat(mFormat, false)

June 30, 2014 - 8:00pm #1

Hi,

I refered to the Dev guide and FAQ to get Camera frame.
I can get correct camera frames correctly.
However the frame rate drops even after I disable it.

Here are my test results

  1. Never enable image format: FPS is stable and over 60.
  2. Enable image format: FPS is 60 during the first few seconds, then becomes 28 ~ 45
  3. Enable image format then disable: FPS is about  45 ~ 57

Please see if we can make case 3 have the same good performance as Case 1.
 

Thanks
Jack

 

Below are my code to dynamic enable/disable image format

using UnityEngine;
using System.Collections;
using System.Runtime.InteropServices;

public class ImageFormat : MonoBehaviour, ITrackerEventHandler {
	
	//private Image.PIXEL_FORMAT mFormat = Image.PIXEL_FORMAT.GRAYSCALE;
	private Image.PIXEL_FORMAT mFormat = Image.PIXEL_FORMAT.RGB565;
	
	private int mRequestFrameNum = 0;
	private bool mRegisteredFormat = false;
	private bool mQCARInitialized = false;

	private bool mToggleEnable = true;
	
	// Use this for initialization
	void Start () {
		QCARBehaviour qcar = (QCARBehaviour)FindObjectOfType(typeof(QCARBehaviour));
		if (qcar) {
			qcar.RegisterTrackerEventHandler(this);
		}
		else {
			Debug.LogError("Could not find QCARBehaviour (i.e. ARCamera) in the scene");
		}
	}
	
	// Implementing OnInitialized() 
	// of ITrackerEventHandler interface
	public void OnInitialized() {
		mQCARInitialized = true;
		mRegisteredFormat = false;
		RegisterFormat ();
	}
	
	// Implementing OnTrackablesUpdated() 
	// of ITrackerEventHandler interface
	public void OnTrackablesUpdated() {
	}
	
	// This is called whenever the app gets paused
	void OnApplicationPause (bool paused) {
		if (paused) {
			// invalidate registered format if app has been paused
			mRegisteredFormat = false;
		} else {
			RegisterFormat();
		}
	}
	
	private void RegisterFormat() {

		if (!mRegisteredFormat) { 
			//first time update or first resume after pause
			//see OnApplicationPaused() above
			CameraDevice.Instance.SetFrameFormat(mFormat, false);
			if (CameraDevice.Instance.SetFrameFormat(mFormat, true)) {
				mRegisteredFormat = true;
				mToggleEnable = true;
			}
			else {
				mFormat = Image.PIXEL_FORMAT.UNKNOWN_FORMAT;
				Debug.LogError("Failed to enale");
			}
		}
	}
	
	// Update is called once per frame
	void Update () {
		// Skip if QCAR has not been initialized yet
		if (mQCARInitialized) {
			if (mRegisteredFormat && mToggleEnable) {
				Image img = CameraDevice.Instance.GetCameraImage(mFormat);
				if (img != null) {
					Debug.Log("Got camera frame. width=" + img.Width);
				}
				else {
					Debug.LogWarning("Failed to get camera frame");
				}
			}
		}
	}
	
	void OnGUI() {
		int bottonw = Screen.width / 2;
		int bottonh = Screen.height / 8;
		GUIStyle style = new GUIStyle();
		style.fontSize = 45;
		if (GUI.Button(new Rect(Screen.width / 2 - bottonh / 2, Screen.height - bottonh, bottonh, bottonh), mToggleEnable.ToString(),
		               style)) {
			mToggleEnable = !mToggleEnable;
			Debug.Log("Toggle. Now is " + mToggleEnable);
			SetFrameFormat(mToggleEnable);
		}
	}
	
	private bool SetFrameFormat(bool enable) {
		if (CameraDevice.Instance.SetFrameFormat(mFormat, enable)) {
			Debug.Log("SetFrameFormat " + enable + " succeed.");
			return true;
		} else {
			Debug.LogWarning("SetFrameFormat " + enable + " fail.");
			return false;
		}
	}
}

Code to measure FPS

// Attach this to a GUIText to make a frames/second indicator.
//
// It calculates frames/second over each updateInterval,
// so the display does not keep changing wildly.
//
// It is also fairly accurate at very low FPS counts (<10).
// We do this not by simply counting frames per interval, but
// by accumulating FPS for each frame. This way we end up with
// correct overall FPS even if the interval renders something like
// 5.5 frames.
 
var updateInterval = 0.5;
 
private var accum = 0.0; // FPS accumulated over the interval
private var frames = 0; // Frames drawn over the interval
private var timeleft : float; // Left time for current interval
 
function Start()
{
    if( !guiText )
    {
        print ("FramesPerSecond needs a GUIText component!");
        enabled = false;
        return;
    }
    timeleft = updateInterval;  
}
 
function Update()
{
    timeleft -= Time.deltaTime;
    accum += Time.timeScale/Time.deltaTime;
    ++frames;
 
    // Interval ended - update GUI text and start new interval
    if( timeleft <= 0.0 )
    {
        // display two fractional digits (f2 format)
        guiText.text = "FPS - " + (accum/frames).ToString("f2");
        timeleft = updateInterval;
        accum = 0.0;
        frames = 0;
    }
}

 

Frame rate drops even after SetFrameFormat(mFormat, false)

July 7, 2014 - 6:33pm #3

Thank you AlessandroB.

I tried your suggestion and found that "disable and re-enable the QCARBehaviour" makes camera close and start over again.
For our application, it's not good to have a few black frames and we will also need to modify our code to keep user defined target.

If you see this is purely memory garbage collection issue, we may close the ticket.
I'll work on this later when performance issue becomes critical.

Jack
 

Frame rate drops even after SetFrameFormat(mFormat, false)

July 4, 2014 - 1:20am #2

Hi, 

the code you used is correct (it appears you have adapted it from one of our Knoweldge Base snippets), however, in order to restore the original frame rate in case 3, you also need to disable and re-enable the QCARBehaviour (as this will internally perform some additional cleanup);

so, instead of just calling (to reset the format):

CameraDevice.Instance.SetFrameFormat( mFormat, false );

you can call this more complete code to make a "full cleanup". This should ensure proper restoring of the original frame rate (as I also verified over here):

private void DisableAllFormats() {
  CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.RGBA8888, false);
  CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.GRAYSCALE, false);
  CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.RGB565, false);
  CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.YUV, false);
  CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.RGB888, false);

  // Reset QCARbehaviour (i.e. camera and trackers)
  QCARBehaviour qcar = (QCARBehaviour)FindObjectOfType(typeof(QCARBehaviour));
  qcar.enabled = false;
  qcar.enabled = true;
}

 

So, if you just want to reset to a default original state, and get back to your original full frame rate, just call the function DisableAllFormats().

If you want to then set again a specific format, the best is to call the following 2 lines in sequence:

DisableAllFormats();

CameraDevice.Instance.SetFrameFormat( YOUR_DESIRED_FORMAT, true );

 

Log in or register to post comments