Log in or register to post comments

Corner Detection on Video Background

June 29, 2012 - 3:01pm #1

Hi! I want to make a corner detection on the video background and create a virtual button on the detected corners. I have seen corner detection on openCV, but l am not sure if there is a way to use openCV with Vuforia. So, if someone could give me some guidance with this it would be highly appreciated. Many thanks!!

 

Regards,

Matias

Corner Detection on Video Background

July 3, 2012 - 1:58pm #13

I haven't worked w/ CV on iOS. Check on some of the iOS and OpenCV forums. They'll know what's possible.

Corner Detection on Video Background

July 3, 2012 - 1:02pm #12

DavidBeard wrote:

I think that you'd have to wrap the FastCV libraries as a native Unity plugin, which will require Unity Pro. Also FastCV currently only support Android. But if these two requirements don't dissuade you, I'll look into the best approach to integrating FastCV.

What are you trying to accomplish in the long run? Do you need a full set of CV libraries, or something more limited?

I need this to work mainly on iOS (iPad), do you know if it is possible with another lib?

Corner Detection on Video Background

July 2, 2012 - 6:13pm #11

I think that you'd have to wrap the FastCV libraries as a native Unity plugin, which will require Unity Pro. Also FastCV currently only support Android. But if these two requirements don't dissuade you, I'll look into the best approach to integrating FastCV.

What are you trying to accomplish in the long run? Do you need a full set of CV libraries, or something more limited?

Corner Detection on Video Background

July 2, 2012 - 6:01pm #10

David, do you have any insight on how to get FastCV to work with Vuforia as plugins in Unity? 

 

DavidBeard wrote:

You can also obtain the pixels of the Image directly, as a byte array, using the following code which is implemented as a tracker event handler - i.e. attach it to the ARCamera. using UnityEngine; using System.Collections; public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler { private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888; private bool m_RegisteredFormat = false; private bool m_LogInfo = true; void Start() { QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour)); if (qcarBehaviour) { qcarBehaviour.RegisterTrackerEventHandler(this); } } public void OnTrackablesUpdated() { if (!m_RegisteredFormat) { CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true); m_RegisteredFormat = true; } if (m_LogInfo) { CameraDevice cam = CameraDevice.Instance; Image image = cam.GetCameraImage(m_PixelFormat); if (image == null) { Debug.Log(m_PixelFormat + " image is not available yet"); } else { string s = m_PixelFormat + " image: \n"; s += " size: " + image.Width + "x" + image.Height + "\n"; s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n"; s += " stride: " + image.Stride; Debug.Log(s); m_LogInfo = false; } } } } Also you might want to look at Qualcomm's FastCV SDK, which is optimized for mobile devices - https://developer.qualcomm.com/mobile-development/mobile-technologies/computer-vision-fastcv

Corner Detection on Video Background

July 2, 2012 - 2:39pm #9

You can also obtain the pixels of the Image directly, as a byte array, using the following code which is implemented as a tracker event handler - i.e. attach it to the ARCamera.

using UnityEngine;
using System.Collections;

public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
{
private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
private bool m_RegisteredFormat = false;
private bool m_LogInfo = true;

void Start()
{
QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
if (qcarBehaviour)
{
qcarBehaviour.RegisterTrackerEventHandler(this);
}
}

public void OnTrackablesUpdated()
{
if (!m_RegisteredFormat)
{
CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
m_RegisteredFormat = true;
}

if (m_LogInfo)
{
CameraDevice cam = CameraDevice.Instance;
Image image = cam.GetCameraImage(m_PixelFormat);

if (image == null)
{
Debug.Log(m_PixelFormat + " image is not available yet");
}
else
{
string s = m_PixelFormat + " image: \n";
s += " size: " + image.Width + "x" + image.Height + "\n";
s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
s += " stride: " + image.Stride;
Debug.Log(s);

m_LogInfo = false;
}
}
}
}

Also you might want to look at Qualcomm's FastCV SDK, which is optimized for mobile devices - https://developer.qualcomm.com/mobile-development/mobile-technologies/computer-vision-fastcv

Corner Detection on Video Background

July 2, 2012 - 2:13pm #8

Hello,

Take a look at the BackgroundTextureAccess sample. You can render the camera image using a Texture2D object. Follow the Readme.txt instructions in this sample to render the camera image using the ARCamera. "Deactivate the BackgroundCamera (and its child object) and activate the VideoBackground child of the ARCamera. Everything should now render inside Unity as expected.

Thank you,

-Peter

Corner Detection on Video Background

July 2, 2012 - 12:49pm #7

How do render that texture back in the app? I want to process the image but still render the video

Corner Detection on Video Background

July 2, 2012 - 11:17am #6

Hello,

You can use the Unity RenderTexture to get the camera image as a texture. Set the camera to render to a texture instead of to the screen (this can be found in the Target Texture field in the Inspector). I think this will get you what you want. Please note that the background camera image will no longer appear on screen after making this change but the augmentation will still appear and track to the target.

Thank you,

-Peter

Corner Detection on Video Background

July 2, 2012 - 10:54am #5

What I want to do is to detect the corners using opencvsharp, so it would be nice to know where I can get the image from the camera, pass it to opencvsharp and it will return the image modified with the corners detected. 

Corner Detection on Video Background

July 2, 2012 - 8:50am #4

Hi Matiasjaure

At present there are no guides for how to integrate these two SDKs, but theoretically there is no reason why they cannot integrate.

Just so I understand correctly is it that you want to understand where the corners are for an image target?

 

N

Corner Detection on Video Background

July 2, 2012 - 7:04am #3

ina wrote:

I would like to see some support articles on integrating Qualcomm's FastCV with Vuforia AND Unity as well! 

Do you know if it is possible?

Corner Detection on Video Background

July 1, 2012 - 2:15pm #2

I would like to see some support articles on integrating Qualcomm's FastCV with Vuforia AND Unity as well! 

Log in or register to post comments