Hello,

Apologies, but Vuforia 7.2 does not provide APIs to access the source image of a User Defined Target.

One possible approach would be to capture the camera frames just prior to creating a User Defined Target: https://library.vuforia.com/content/vuforia-library/en/articles/Solution/Working-with-the-Camera.html#How-To-Obtain-HD-Camera-Frames (see "How To Access the Camera Image in Unity").

Thanks,

Vuforia Support

I know that this subject has been asked in like 100 threads but they are all old and no documentation is available about accessing images without augmentation. 

I run the sample app which includes a user-defined sample which calls this method:



public void BuildNewTarget()

    {

        if (m_FrameQuality == ImageTargetBuilder.FrameQuality.FRAME_QUALITY_MEDIUM ||

            m_FrameQuality == ImageTargetBuilder.FrameQuality.FRAME_QUALITY_HIGH)

        {

            // create the name of the next target.

            // the TrackableName of the original, linked ImageTargetBehaviour is extended with a continuous number to ensure unique names

            string targetName = string.Format("{0}-{1}", ImageTargetTemplate.TrackableName, m_TargetCounter);

            // generate a new target:

            m_TargetBuildingBehaviour.BuildNewTarget(targetName, ImageTargetTemplate.GetSize().x);

        }

        else

        {

            Debug.Log("Cannot build new target, due to poor camera image quality");

            if (m_QualityDialog)

            {

                StopAllCoroutines();

                m_QualityDialog.GetComponent<CanvasGroup>().alpha = 1;

                StartCoroutine(FadeOutQualityDialog());

            }

        }

    }

The issue is that, I am not sure how to access the source file image. 

Thanks for your support in advance.

Hello,

These are good points. Thank you for the detailed feedback.

OnTrackablesUpdated is not guaranteed to happen every frame. In fact, if the app runs at 60fps but the device camera only captures at 30fps, you only get it every other frame. Thus, it will not be easy to implement something that needs to happen every frame - only after Vuforia finishes updating.

We'll look into changing this for a future release.

Thanks.

Hi Medabit,

Thank you so much for such a thorough reply, it is hugely helpful to know these details.

I’ll explain a bit more my use case and my current setup:

In my Scene (among other things) I have an ARCamera, a VuMark and an empty game object that I use as a “processing manager” - let’s call it ManagerObject. ManagerObject has a script attached - let’s call it ProcessCameraImage.

During the Start() of ProcessCameraImage I register a TrackablesUpdated callback, during which the camera image is accessed and subsequently processed. The image processing is done only on a specific region determined by the VuMark’s position with respect to the ARCamera. The result of this processing is stored.

During the Update() of ProcessCameraImage, the result of the image processing of the current frame (computed during the TrackablesUpdated callback) is compared to the processing result of the previous frame, and a “decision" is made based on that.

In my current setup, I think what’s most critical is:

* Having the guarantee that the TrackablesUpdated callback is called in *every* single Update cycle, since otherwise the decision making I do during Update would be messed up. Do you think this is something I can rely upon even if you change things in the future?

* Understanding the execution order of the Update() during which the trackables are updated, and the Update() of my ProcessCameraImage script. As far as I know the order in which Unity executes Update functions is arbitrary although it can be enforced in the Project Settings if necessary. Is anything done internally in Vuforia with respect to controlling this execution order at all? Not sure if it’s even possible but it’s worth asking :)

Regarding your question, I don’t think my use case falls into that category, I wish I could be more helpful. The only reason I can think of that being necessary would be if I wanted to compute something physics-related during FixedUpdate that had to match the state of the trackables. If you’d want to share a use case yourself that would be super interesting to read.

Thanks a million again for your help!

Hello,

martejpad wrote:

2. When is OnTrackablesUpdated() actually called? If the event "all trackables updated" is registered, is the function called right before the following Update()? (This would match the way current Input Events in Unity are handled, according to the execution order)

The VuforiaBehaviour listens for Update() itself, and updates all trackables and the camera image from that callback. When it is done, it invokes all OnTrackablesUpdated callbacks in user code.

But, in the future, we cannot guarantee that OnTrackablesUpdated will be called during an Update() as we might change it. The best thing to rely upon is that the state of all trackables have been updated in the scene when you receive the OnTrackablesUpdated callback.

Question: Do you have a use case that would required Vuforia to finish updating all trackable updates before Update()? We are looking more closely at this requirement and your input would be helpful.

Thanks

Hello,

martejpad wrote:

1. If instead of wanting to access the camera image that matches the current frame I'm interested in accessing the camera image as fast as possible (i.e. accessing as many frames as possible), even if that means it's not the frame currently being displayed, how would I go about it? Should I simply access the camera image in Unity's Update() function?

Vuforia processes camera frames as received by the OS in real time. Thus, there is no way to access frames before they are presented by Vuforia. If you wish to store the camera frames so that you can go back to previous ones, then this would be handled outside of the SDK.

Note that when the Vuforia SDK is running (and the camera is initialized), no other processes will be able to access the camera.

martejpad wrote:

2. When is OnTrackablesUpdated() actually called? If the event "all trackables updated" is registered, is the function called right before the following Update()? (This would match the way current Input Events in Unity are handled, according to the execution order)

I'll do a bit more research into #2 and respond back.

Thanks.