Log in or register to post comments

Slow rendering with Cloud Recognition

January 18, 2019 - 10:01am #1

Hi! My experience with Cloud Reco shows that recognition works quickly only if you move your phone closer/further to the target, but regular users will not do so because they do not know that this will be faster. Could you explain how can I speed up the process of recognizing a target?

The target itself is ALWAYS recognized quickly (almost immediately the message "Successfully created new trackable '% s' with rating '% d'" goes to the log).

BUT in the method “- (void)renderFrameWithState: projectMatrix:” state.getTrackableResults().size() still equals 0

it increases much later when we start moving the phone or just with a long delay (sometimes it even reaches 1 minute)

It turns out that the target is recognized quickly, but nothing is shown until in the method “- (void) renderFrameWithState: projectMatrix:” state.getTrackableResults().size() is increased

How can this be accelerated?

Slow rendering with Cloud Recognition

January 24, 2019 - 2:59am #7

Quote:

You're not using getTrackableResults().size(), which is the test shown in the sample and you referred to as the test. Additionally, you seem to be incorrectly using TrackableResultList:

here is this piece of code

TrackableResultList trackableResultList = state.getTrackableResults();
for (TrackableResult result : trackableResultList)
{
    modelMatrix = Tool.convertPose2GLMatrix(result.getPose());

    mActivity.runOnUiThread(new Runnable() {
        @Override
        public void run() {
            if(!rendered) { // by default rendered is false
                rendered = true;
                Toasty.success(mActivity, "TrackableResultList > 0").show();
            }
        }
    });

    if (result.isOfType(ImageTargetResult.getClassType()))
    {
        mActivity.stopFinderIfStarted();

        // Renders the augmentation
        renderModel(projectionMatrix, devicePoseMatrix.getData(), modelMatrix.getData());

        SampleUtils.checkGLError("CloudReco renderFrame");
    }
}

If the Toast is displayed, means that TrackableResultList > 0

 

Could you test on the same marker as in video?

camera should be immediately close to the marker, phone should not move (As in the video) and it should be first recognition of the marker after vuforia started (because cloudRecoResult created once for the same marker)

I attached the whole project 

open with Android Studio

"Open an existing Android Studio project" > "...vuforia/samples/VuforiaSamples-8-0-10"

the keys are already set up, just hover close to the marker with no movement

Thanks

AttachmentSize
Package icon vuforia.zip93.99 MB
Image icon target.png749.01 KB

Slow rendering with Cloud Recognition

January 23, 2019 - 4:08pm #6

Hello,

We just did a similar test and did not observe the issue.

What I don't understand is how you're using the APIs for your "toastie" message that isn't displayed until 0:50:

Toasty.success(mActivity, "TrackableResultList > 0").show();

You're not using getTrackableResults().size(), which is the test shown in the sample and you referred to as the test. Additionally, you seem to be incorrectly using TrackableResultList:

https://library.vuforia.com/content/vuforia-library/en/reference/java/classcom_1_1vuforia_1_1TrackableResultList.html

Thanks,

Vuforia Engine Support

 

Slow rendering with Cloud Recognition

January 23, 2019 - 11:47am #5

I want to work with the marker immediately after its recognition (at 0:13), not at 0:50.

I can't understand why state.getTrackableResults().size(); (CloudRecoRenderer.java) becomes greater than zero only after the second query to cloud reco server (0:50).

In the video we see that at 0:13 the server returned the correct answer, but for some reason the library still calls function

public void renderFrame(State state, float[] projectionMatrix)

with state.getTrackableResults().size() = 0

It will be increased only with the next cloud query (0:50), that's what I'm talking about.

I hope for your help.

Slow rendering with Cloud Recognition

January 23, 2019 - 10:31am #4

Hello,

Thanks for the additional information. I'm afraid that I don't understand your use case, and how Vuforia's current behavior is not meeting it.

What I *think* I'm observing in your video:

  1. Cloud target is successfully recognized (~0:13 in video)
  2. Cloud target is being 'tracked' with no augmentation (after 0:13)

At this point, the SDK *will not* execute another cloud query as the target is being actively tracked (per the core sample implementation, as I understand it). The sample's implementation is an example UX, one that limits the number of queries to recognize a target. It is very easy to create a UX that generates an excessive number of unsuccessful cloud queries, which could create a low query to success ratio - something you want to avoid.

Additionally, cloud queries are only triggered when there is a scene change, and the quality of the image frame meets a minimum requirement - see explanation in my previous post.

Later in the sequence it looks like another query is triggered, but I'm not clear why. Note that if the service returns a query with the same target that was already reco'ed, the SDK will not take any action and the reco is ignored.

Again, understanding the use cases/UX you're trying to develop for would be helpful.

Thanks,

Vuforia Engine Support

Slow rendering with Cloud Recognition

January 23, 2019 - 5:40am #3

Thanks for your reply.

The problem is not in recognition (recognition occurs quickly), the problem is precisely in the rendering on the screen after recognition (provided that the phone is almost not moving)

I reproduce this problem on Samsung Galaxy A5, iPhone X, iPhone 6 Plus (I think this behavior is on all devices)

Vuforia 8.0.10 (and Vuforia 7.5.26)

 

I used sample Cloud Recognition, just replaced the keys and added two Toasts (two modal windows (see the video below))

1. CloudReco.java 769-773 lines:

if (cloudRecoResult != null && cloudRecoResult.getTrackingRating() > 0)
{
    Toasty.info(CloudReco.this, "CloudRecoSearchResult: " + result.getTargetName()).show();
    finder.enableTracking(cloudRecoResult);
}

2. CloudRecoRenderer.java 215-225 lines:

modelMatrix = Tool.convertPose2GLMatrix(result.getPose());

mActivity.runOnUiThread(new Runnable() {
    @Override
    public void run() {
        if(!rendered) { // by default rendered is false
            rendered = true;
            Toasty.success(mActivity, "TrackableResultList > 0").show();
        }
    }
});

screen recording and target: https://www.dropbox.com/sh/w5n55givhjbr3t6/AABuV6HTV9xjqEywN0LFRbp1a?dl=0

Slow rendering with Cloud Recognition

January 22, 2019 - 1:28pm #2

Hello,

Are you observing similar, long delays when running our sample app?

If so, please provide:

  • The device model being used for testing: Settings->About phone/device->Model number
  • Vuforia Engine SDK version
  • The Unity Editor version (if applicable)

Additionally, steps to recreate the issue are also very helpful.

Some additional information below that you may find relevant.

Thanks,

Vuforia Engine Support

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

When the SDK Cloud Reco service is enabled, each camera frame is analyzed for two properties:

  1. A "scene change", which determines if what is in the the camera's field of view has changed, and
  2. Frame quality, which determines if an incoming camera frame is in-focus, exhibits no blurring and has sufficient lighting for recognition

If both of these conditions are met, then the SDK service will trigger a "query" to the Cloud Reco server. The server determines if elements of that query's camera frame match a target in the endpoint Cloud Database, and if it does then this triggers a "reco", which in-turn increments the reco count. Unsuccessful queries are not counted against the quota.

This SDK query operation is not normally presented to the application layer, thus it can be difficult to observe when a query is triggered during typical SDK usage. To better help developers understand the SDK's behavior, the Cloud Reco 'filter mode' was introduced in Vuforia 5: https://library.vuforia.com/articles/Solution/Whats-New-in-Vuforia-5. This functionality is intended for diagnostic purposes only, and is not recommended for use in production apps.

The updateSearchResult method accepts a "filter" setting value of either FILTER_NONE (disable filtering), or FILTER_CURRENTLY_TRACKED (enable filtering for targets being tracked) - with the latter being the default behavior of the TargetFinder. By default, Cloud Reco query results are not returned to the application layer for targets that are currently being tracked. If the FILTER_NONE argument is used, these results will be returned, regardless of whether the same target has already been enabled for tracking. The purpose of removing this "filter" is to enable developers to determine if reco event are unexpectedly occurring on the same target due to user behavior (normally due to a scene change due from inadvertent camera movement).

Additionally, inherent network latency can introduce race conditions in which more than one Cloud Recognition event would be logged at the portal. The following is one example:

  1. The user points the camera at a target in the Cloud Database, and a query is sent to the Cloud Reco service.
  2. The user moves the device slightly (with the same target still in the camera’s view), which triggers a second query being sent to the Cloud Reco service (aka a "scene change")
  3. The initial query is processed by the Cloud Reco service, and the service returns a successful reco result. The client handles the response before the second query response returns.
  4. The second query response is received at the client, but the first query has already been handled for this target. Thus, the client discards this result as a duplicate.

In the above example, both queries will be counted as successful recos on the Cloud Database.

Depending upon the use case, we normally recommend that the SDK Cloud Reco service be stopped after a successful reco. This will help to better control when a query occurs. A UX element can be introduced at the app layer if it is desired to trigger another query.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Log in or register to post comments