I haven't worked w/ CV on iOS. Check on some of the iOS and OpenCV forums. They'll know what's possible.

DavidBeard wrote:

I think that you'd have to wrap the FastCV libraries as a native Unity plugin, which will require Unity Pro. Also FastCV currently only support Android. But if these two requirements don't dissuade you, I'll look into the best approach to integrating FastCV.

What are you trying to accomplish in the long run? Do you need a full set of CV libraries, or something more limited?

I need this to work mainly on iOS (iPad), do you know if it is possible with another lib?

I think that you'd have to wrap the FastCV libraries as a native Unity plugin, which will require Unity Pro. Also FastCV currently only support Android. But if these two requirements don't dissuade you, I'll look into the best approach to integrating FastCV.

What are you trying to accomplish in the long run? Do you need a full set of CV libraries, or something more limited?

David, do you have any insight on how to get FastCV to work with Vuforia as plugins in Unity? 

DavidBeard wrote:

You can also obtain the pixels of the Image directly, as a byte array, using the following code which is implemented as a tracker event handler - i.e. attach it to the ARCamera. using UnityEngine; using System.Collections; public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler { private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888; private bool m_RegisteredFormat = false; private bool m_LogInfo = true; void Start() { QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour)); if (qcarBehaviour) { qcarBehaviour.RegisterTrackerEventHandler(this); } } public void OnTrackablesUpdated() { if (!m_RegisteredFormat) { CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true); m_RegisteredFormat = true; } if (m_LogInfo) { CameraDevice cam = CameraDevice.Instance; Image image = cam.GetCameraImage(m_PixelFormat); if (image == null) { Debug.Log(m_PixelFormat + " image is not available yet"); } else { string s = m_PixelFormat + " image: \n"; s += " size: " + image.Width + "x" + image.Height + "\n"; s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n"; s += " stride: " + image.Stride; Debug.Log(s); m_LogInfo = false; } } } } Also you might want to look at Qualcomm's FastCV SDK, which is optimized for mobile devices - https://developer.qualcomm.com/mobile-development/mobile-technologies/computer-vision-fastcv

You can also obtain the pixels of the Image directly, as a byte array, using the following code which is implemented as a tracker event handler - i.e. attach it to the ARCamera.

using UnityEngine;
using System.Collections;

public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
{
private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
private bool m_RegisteredFormat = false;
private bool m_LogInfo = true;

void Start()
{
QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
if (qcarBehaviour)
{
qcarBehaviour.RegisterTrackerEventHandler(this);
}
}

public void OnTrackablesUpdated()
{
if (!m_RegisteredFormat)
{
CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
m_RegisteredFormat = true;
}

if (m_LogInfo)
{
CameraDevice cam = CameraDevice.Instance;
Image image = cam.GetCameraImage(m_PixelFormat);

if (image == null)
{
Debug.Log(m_PixelFormat + " image is not available yet");
}
else
{
string s = m_PixelFormat + " image: \n";
s += " size: " + image.Width + "x" + image.Height + "\n";
s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
s += " stride: " + image.Stride;
Debug.Log(s);

m_LogInfo = false;
}
}
}
}

Also you might want to look at Qualcomm's FastCV SDK, which is optimized for mobile devices - https://developer.qualcomm.com/mobile-development/mobile-technologies/computer-vision-fastcv

Hello,

Take a look at the BackgroundTextureAccess sample. You can render the camera image using a Texture2D object. Follow the Readme.txt instructions in this sample to render the camera image using the ARCamera. "Deactivate the BackgroundCamera (and its child object) and activate the VideoBackground child of the ARCamera. Everything should now render inside Unity as expected.

Thank you,

-Peter

How do render that texture back in the app? I want to process the image but still render the video

Hello,

You can use the Unity RenderTexture to get the camera image as a texture. Set the camera to render to a texture instead of to the screen (this can be found in the Target Texture field in the Inspector). I think this will get you what you want. Please note that the background camera image will no longer appear on screen after making this change but the augmentation will still appear and track to the target.

Thank you,

-Peter

What I want to do is to detect the corners using opencvsharp, so it would be nice to know where I can get the image from the camera, pass it to opencvsharp and it will return the image modified with the corners detected. 

Hi Matiasjaure

At present there are no guides for how to integrate these two SDKs, but theoretically there is no reason why they cannot integrate.

Just so I understand correctly is it that you want to understand where the corners are for an image target?

N

ina wrote:

I would like to see some support articles on integrating Qualcomm's FastCV with Vuforia AND Unity as well! 

Do you know if it is possible?

I would like to see some support articles on integrating Qualcomm's FastCV with Vuforia AND Unity as well!