Log in or register to post comments

Beer label recognition is not working

May 13, 2019 - 11:02am #1

Hello,

We are using Vuforia's CLOUD base to recognize beer label taken through a mobile application by our users.

However, the recognition only works if we use the same photo that is in the CLOUD base.

If we use another photo of the same beer the recognition never works.

Is there any recommendation/tip that you have to give us?

Our app is the same as the App Vivino, but for beers.

Should we upload multiple photos for the same beer? Today we are going up 1 to 1.

Thank you.

Beer label recognition is not working

May 16, 2019 - 2:36pm #4

Hello,

Please understand that I cannot detail how Vivino's app works with Vuforia, but I can provide you with a general overview of an approach:

  1. Capture multiple product label images from different angles, removing all detail other than what appears on the label
  2. Add the same images to a Vuforia Cloud Database with metadata (if needed)
  3. Submit user image to the Vuforia Cloud Database using the Vuforia Web Query API
    • Note that Vuforia Web Query API is a *different* service from Engine's Cloud Reco feature. Web Query APIs are NOT part of the SDK.
  4. Perform additional analysis on the image, if necessary (OCR?)
  5. Return results to user

Looking at your latest uploads, I still believe that environmental features are affecting recognition accuracy. I've attached a marked-up image with the region of interest outlined in red (2019-05-16_13-29-28.png). Note that features outside that area appear, and will attempt to be matched. Thus, they need to be removed.

To show you what I'm talking about here is an example:

  • duvel_orig.png: Image downloaded from Google
  • duvel_cloud.jpg: Same image optimized for Cloud Reco

I suggest you do the same and benchmark recognition performance based upon a similarly modified image of beer labels.

Thanks,

Vuforia Engine Support

AttachmentSize
Image icon 2019-05-16_13-29-28.png202.62 KB
Image icon duvel_orig.png1.41 MB
Image icon duvel_cloud.jpg71.96 KB

Beer label recognition is not working

May 16, 2019 - 11:50am #3

Thanks for the feedback.

Subims 6 photos of the same beer (code 230584), following your recommendation: only with beer label and white background.

See the targets that we climb in the base "Beers-Prod":

230584_rotulo01

230584_rotulo02

230584_rotulo03

230584_rotulo04

230584_inclinada

230584

But even so, recognition when taking pictures is not working.

Are there any more settings we need to make to make beer label recognition work? We are worried because we bought the service of Cloud Reco thinking that it would be enough 1 photo of each beer for recognition to work, but that is not what is happening.

In the case of Vivino, do they upload more than 1 photo per wine?

Thank you.

Beer label recognition is not working

May 14, 2019 - 11:03am #2

Hello,

For the images used for recognizing the label, do they have any other elements present (other than elements from the label itself) in the background? Note that the Cloud Reco service will use features from the *entire image* to match. Thus, if there is something in the background of the current camera image that does not match the uploaded one, the recognition may fail.

I took a look at some of the images in your Cloud Database and this seems to be the case. I see several examples where your uploaded images have elements that are not needed for recognition.

Here's what I suggest for your use case:

  1. Use more than one image of the label as they appear on the bottle. The curvature (rather than a flat 2D image) is important as it represents how the image appears in real life.
  2. Only include the bottle label in the image. Remove the bottle and any background elements by turning them to white

Here is some guidance on optimizing targets: https://library.vuforia.com/articles/Solution/Optimizing-Target-Detection-and-Tracking-Stability

Thanks,

Vuforia Engine Support

Log in or register to post comments