Log in or register to post comments

Camera cropped with version 3

October 16, 2014 - 2:49am #1

The camera image has been dramatically cropped with verison 3.0 of the sdk.

I attached two screenshots of our app, and you can clearly see that the four boxes I placed at each border are almost invisibile in version 3

Please note that to be sure that it was not a screen problem, I divided the render zone by 2:

    config.mSize.data[0] /= 2;
    config.mSize.data[1] /= 2;
    QCAR::Renderer::getInstance().setVideoBackgroundConfig(config);

This problem only occurs on ios8 devices.

Is this problem know ?

AttachmentSize
Image icon Image-2.8.jpg50.86 KB
Image icon Image-3.0.jpg67.87 KB

Camera cropped with version 3

October 22, 2014 - 5:17am #20

Correct. 

Of course, since you're already playing with the configureVideoBackground code and you're familiar with the way it works, one option is to just downscale a bit the image so that it appears less "zoomed'; however, as the maths suggest, this can only be done either at the cost of introducing black side bands around the camera image or to introduce an aspect ratio distortion in the rendered camera image. 

Otherwise, the "general" recommendation otherwise remains the same, i.e. to use Mode Speed and get the same (lower) resolution that you were getting in SDK 2.8.

 

Camera cropped with version 3

October 22, 2014 - 4:59am #19

Alessandro,

I did test the speed mode for the getVideoMode, I was not aware that there was a selectVideoMode that needed to be changed also, sorry for that. By the way, I dont understand the use of the getVideoMode parameter, if it's forced to be equal to the select.

So, to conclude on that, refering to

https://developer.vuforia.com/forum/ios/sdk-version-3-0-5-retina-resolution-problem

since the (2) is not relevant, that means that this problem is out of the vuforia scope and I can't hope for a resolution in a next release ?

Camera cropped with version 3

October 22, 2014 - 4:12am #18

Augment, thanks for confirming this:

    When I set and get the video mode with coherent values, I get the correct ratio

    So, the speed mode works, like 2.8.

    The quality mode is zoomed by 40% on the ipad3.
 

 

It's a bit unfortunate that I had already asked to test the MODE_OPTIMIZE_SPEED already in my first post (please read again my first post), and apparently you did not do it at that time, and you only did it now.

My first post reads:

One way around this is to choose camera mode MODE_OPTIMIZE_SPEED, using the selectVideoMode method of the CameraDevice class, during camera initialization:

https://developer.vuforia.com/resources/api/class_q_c_a_r_1_1_camera_device

That mode will give you similar results to the ones you were seeing in SDK 2.x

and at that time you replied:

By the way, I have exactly the same border loss in speed mode

 

So, this brings me back to what I was originally saying in my early posts, i.e., the reason is that SDK 3.0 with Quality and Default MODE is using HD resolution (while SDK 2.8 is using 640 x 480);   i.e., due to the reasons described int he other thread I was referring to (iPad known issue):

https://developer.vuforia.com/forum/ios/sdk-version-3-0-5-retina-resolution-problem

 

the MODE_OPTIMIZE_QUALITY and MODE_DEFAULT will result in "zoomed" camera image (exactly as you observed). The solution is to use MODE_OPTIMIZE_SPEED.

 

The fact that SDK 2.8 was not producing such a "zoomed" camera image is simply because 2.x was using a resolution of 640x480 in all 3 MODES.

 

 

Camera cropped with version 3

October 22, 2014 - 4:02am #17

Alessandro,

Yes, we dont set the viewport (at least for these tests).
 
Here is the code that set the camera mode:
 
        if (!QCAR::CameraDevice::getInstance().init(QCAR::CameraDevice::CAMERA_DEFAULT))
            AR_THROW_EXCEPTION_TYPED(ProblemType::QCar, "Impossible to initialize camera. You may have to reboot your device.");
        if (!QCAR::CameraDevice::getInstance().selectVideoMode(QCAR::CameraDevice::MODE_DEFAULT))
            AR_THROW_EXCEPTION_TYPED(ProblemType::QCar, "Impossible to set video mode.");
        if (!QCAR::CameraDevice::getInstance().start())
            AR_THROW_EXCEPTION_TYPED(ProblemType::QCar, "Impossible to start camera. You may have to reboot your device.");
        if (!QCAR::CameraDevice::getInstance().setFocusMode(QCAR::CameraDevice::FOCUS_MODE_CONTINUOUSAUTO))
            Logging::warn("Impossible to set autofocus.");
 
 
When I set and get the video mode with coherent values, I get the correct ratio, and the behavior of the logs are:
 
Quality:
[INFO ] VideoBackgroundConfig: size: 768,1365
[INFO ] VideoMode:w=1280 h=720
[INFO ] width=   1536 height=   2048
[INFO ] ViewPort: X,Y: 384,341 Size X,Y:768,1365
 
Speed:
[INFO ] VideoBackgroundConfig: size: 768,1024
[INFO ] VideoMode:w=640 h=480
[INFO ] width=   1536 height=   2048
[INFO ] ViewPort: X,Y: 384,512 Size X,Y:768,1024
 
 
 
So, the speed mode works, like 2.8.
The quality mode is zoomed by 40% on the ipad3.

Camera cropped with version 3

October 22, 2014 - 2:15am #16

Forgot to mention:

could you also share the code in which you select the camera video mode ?

(using the QCAR::CameraDevice::getInstance().selectVideoMode(  ) function ?)

i.e. some code like:

// initialize the camera

    if (! QCAR::CameraDevice::getInstance().init(camera)) {

        [self NSErrorWithCode:-1 error:error];

        return NO;

    }

    

    // select the default video mode

    if(! QCAR::CameraDevice::getInstance().selectVideoMode(QCAR::CameraDevice::MODE_DEFAULT)) {

        [self NSErrorWithCode:-1 error:error];

        return NO;

    }

    

    // start the camera

    if (!QCAR::CameraDevice::getInstance().start()) {

        [self NSErrorWithCode:-1 error:error];

        return NO;

    }

 
In the code above, make sure to replace MODE_DEFAULT with MODE_OPTIMIZE_SPEED or MODE_OPTIMIZE_QUALITY consistently with the mode that you use in the configureVideobackground getVideoMode.
 

I see you retrieve the MODE_OPTIMIZE_QUALITY from the camera using the getVideoMode(), 

QCAR::VideoMode videoMode = QCAR::CameraDevice::getInstance().getVideoMode(QCAR::CameraDevice::MODE_OPTIMIZE_QUALITY);

however, you must also make sure that the selectVideoMode (using the same MODE) has been successfully applied BEFORE the camera is started (see camera initialization code).

Camera cropped with version 3

October 22, 2014 - 2:07am #15

Thanks.

The code snippet makes sense,and the dimensions logged indeed reflect the maths used (including the divide by two).

Just one last little thing to check before I can proceed to repro the same exact scenario on the iPad 3, could you confirm that you are not setting the OpenGL viewport in a custom way in your application code ?

I.E. is there any call to glViewport and/or glScissor with arguments (parameters) that depend somehow on the values of the video background config position and size ?

Thanks much.

 

Camera cropped with version 3

October 22, 2014 - 12:55am #14

Hi Alessandro,

 

Here is the code:

    QCAR::onSurfaceChanged(screenWidth, screenHeight);

    // Configure the video background
    QCAR::VideoMode videoMode = QCAR::CameraDevice::getInstance().getVideoMode(QCAR::CameraDevice::MODE_OPTIMIZE_QUALITY);
    QCAR::VideoBackgroundConfig config;
    config.mEnabled = true;
    config.mSynchronous = true;
    config.mPosition.data[0] = 0.0f;
    config.mPosition.data[1] = 0.0f;

    if (screenWidth < screenHeight) {
        config.mSize.data[0] = (int) (videoMode.mHeight * (screenHeight / (float)videoMode.mWidth));
        config.mSize.data[1] = (int) screenHeight;

        if (config.mSize.data[0] < (int) screenWidth) {
            config.mSize.data[0] = (int) screenWidth;
            config.mSize.data[1] = (int) (screenWidth * (videoMode.mWidth / (float)videoMode.mHeight));
        }
    }
    else {
        config.mSize.data[0] = (int) screenWidth;
        config.mSize.data[1] = (int) (videoMode.mHeight * (screenWidth / (float)videoMode.mWidth));

        if (config.mSize.data[1] < (int) screenHeight) {
            config.mSize.data[0] = (int) (screenHeight * (videoMode.mWidth / (float)videoMode.mHeight));
            config.mSize.data[1] = (int) screenHeight;
        }
    }

    config.mSize.data[0] /= 2;
    config.mSize.data[1] /= 2;

    QCAR::Renderer::getInstance().setVideoBackgroundConfig(config);

 

 

And here is the logs:

Quality:

[INFO ] VideoBackgroundConfig: size: 768,1365
[INFO ] VideoMode:w=1280 h=720
[INFO ] width=   1536 height=   2048
[INFO ] ViewPort: X,Y: 384,341 Size X,Y:768,1365

 

Speed:

[INFO ] VideoBackgroundConfig: size: 768,1024
[INFO ] VideoMode:w=640 h=480
[INFO ] width=   1536 height=   2048
[INFO ] ViewPort: X,Y: 384,512 Size X,Y:768,1024

 

 

Regards,

Camera cropped with version 3

October 21, 2014 - 11:57am #13

Thanks for this test. The speed vs quality screenshots are helpful.

just to be on the safe side, would you mind sharing your "custom" version of the configureVideoBackground function ? so I can try executing the same code.

Also, if you check the Logs, what is the video size and the screen size logged by the configureVideoBackground when using SPEED vs. when using QUALITY?

Here I'm referring to this sample code:

NSLog(@"VideoBackgroundConfig: size: %d,%d", config.mSize.data[0], config.mSize.data[1]);

    NSLog(@"VideoMode:w=%d h=%d", videoMode.mWidth, videoMode.mHeight);

    NSLog(@"width=%7.3f height=%7.3f", viewWidth, viewHeight);

    NSLog(@"ViewPort: X,Y: %d,%d Size X,Y:%d,%d", viewport.posX,viewport.posY,viewport.sizeX,viewport.sizeY);

 

 

 

 

Camera cropped with version 3

October 21, 2014 - 9:03am #12

Alessandro,

 

Here is a test on a 4th device: an iPad3 (retina) with ios8.

The two screenshot are taken witht the size divided by 2 again, one with MODE_OPTIMIZE_SPEED, the other with MODE_OPTIMIZE_QUALITY.

On this device, the crop is maximum (this is the original device where I detected the bug).

 

As you can see, there is no border differences, only one in ratio.

AttachmentSize
Image icon retina_ios8_speed.jpg240.17 KB
Image icon retina_ios8_quality.jpg310.07 KB

Camera cropped with version 3

October 21, 2014 - 8:26am #11

You are right, it's not an ios8 problem

Thanks for confirming this.

 

What I do not understand is that you say that it's not depending of the vuforia implementation (since I remove the screen size crop problem), but if so, why is there a difference between vuforia 2 and 3 ?

My statement refers to the fact that the issue could originate from 2 factors (or a combination of both):

- the configureVideoBackground function (this is part of the sample code): however, you already ruled out this, as you use custom code by dividing the size by two, so we can exclude this

- the SDK 3.0 is selecting and using an HD camera image (e.g. 1280 x 720 for example) and this image is partially "zoomed" with respect to other resolution;   whereas SDK 2.x would select a lower resolution (on high-end retina devices);   this is also why I originally asked you if you can make a test by selecting MODE_OPTIMIZE_SPEED, because this should deliver a different "zoom" factor.  Concerning this, you said previously that you did try using CAMERA_MODE_SPEED, however, are you sure you successfully enabled that mode ? Perhaps worth double- checking, as normally this should do the trick

 

 

Camera cropped with version 3

October 21, 2014 - 7:59am #10

Alessandro,

 

You are right, it's not an ios8 problem

I attached 6 screenshots to this mail, showing that the problem comes from vuforia 3.

The strange thing is, that the same screenshots on a non retina device on ios6 is always "cropped" !

So, the only "good" version is vuforia 2 on an ios 7 or 8.

 

It's indeed probably the same problem as described here:

https://developer.vuforia.com/forum/ios/sdk-version-3-0-5-retina-resolution-problem

What I do not understand is that you say that it's not depending of the vuforia implementation (since I remove the screen size crop problem), but if so, why is there a difference between vuforia 2 and 3 ?

AttachmentSize
Package icon vuforia_test.zip1.55 MB

Camera cropped with version 3

October 21, 2014 - 6:33am #9

Augment,

The screenshots you provided only show a difference between Vuforia 2.8 and 3.0

(NOT a difference between iOS 7 and iOS8).

I asked a comparison between iOS 7 and iOS 8 in reference to your statement:

"please note that the problem only occurs on ios8."

Would you be able to bring a few screenshots that show the  iOS7 vs iOS8 difference (on the same device, for example on an iPad 4 running iOS7 and a similar model of iPad 4 running on iOS8) ?

I'm asking also because the issue you show in the last Vuforia Sample Screenshot (sample_vuforia_3_0.jpg)  is a known issue with some iOS high-end devices using HD resolution,a s I was saying in my first post,(such as iPad Air / iPad 4), which is not related to the use of iOS 8 (i.e. it also occurs on iOS previous versions, as we verified);

this issue has been already discussed in this Forum thread:

https://developer.vuforia.com/forum/ios/sdk-version-3-0-5-retina-resolution-problem

 

So, maybe the issue with you App is not necessarily the same issue that you show in the sample screenshot "sample_vuforia_3_0.jpg". (see forum thread linked above)

And, maybe your App issue is related to iOS 8 (but the Vuforia Sample screenshot issue is NOT), in which case we need to see a comparative screenshot of the same App running on the same device model but on 2 different iOS versions (7 vs. 8).

 

 

 

 

 

 

Camera cropped with version 3

October 21, 2014 - 6:06am #8

Alessandro,

No, the problem also occurs with samples in portrait mode.

Please see the attached screenshots: samples have exaclty the same cropped view, and augment with vuforia 2.8 is good.

 

AttachmentSize
Image icon augment_vuforia_2_8.jpg405.2 KB
Image icon sample_vuforia_3_0.jpg605.49 KB

Camera cropped with version 3

October 21, 2014 - 5:43am #7

Right. So, it must be the iOS 8 issue then

(now looking again at your original screenshots, I see that you use black borders, which rules out the screen cropping indeed; initially I though you had manually added the black borders to the Screenshot in order to show the Augment logo,   I did not realize that it was the actual device screen).

 

For ios8, yeah, probably a migration problem, I agree, but since the behavior is the same on the samples applications,

I guess it's on the vuforia lib side.

Probably the issue is with the Vuforia Samples code (not with the Library itself), as described here:

https://developer.vuforia.com/forum/ios/ios-8-xcode-6-and-vuforia-30-compatibility

 

However, I would expect that you observe the issue on iOS 8 samples only when using the Books sample in landscape orientation;

as far as the other Vuforia Samples are concerned, they should look OK on iOS8 (as we observed also in our tests), simply because they are all locked in Portrait mode.

So, If you run any of the samples, and you see a clear difference between a device running on iOS 7 and the same device model running on iOS 8, could you post some comaprative screenshots here (indicating which device seems to have changed behaviour between iOS 7 and iOS 8 with the Vuforia samples)....?

 

 

Camera cropped with version 3

October 21, 2014 - 4:57am #6

Alessandro,

 

I know the `configureVideoBackground`, and if you look at my first post, you can see that I divided by 2 the rendering size, to avoid this screen cropping.

(Do you have access to the two screenshots of my first post ?)

So, this is definitly not a screen cropping problem.

 

For ios8, yeah, probably a migration problem, I agree, but since the behavior is the same on the samples applications,

I guess it's on the vuforia lib side.

 

Camera cropped with version 3

October 21, 2014 - 3:56am #5

I dont understand your mention of the screen ratio, since this is handled in the function

- (void)configureVideoBackgroundWithViewWidth:(float)viewWidth andHeight:(float)viewHeight

Why would you need to crop anything outside it ?

Thae configureVideoBackground function will set the video configuration indeed and will produce some "cropping", as you can see from the code below (which is extracted from the sample code function configureVideoBackground...)

// Calculate the viewport for the app to use when rendering

    viewport.posX = ((viewWidth - config.mSize.data[0]) / 2) + config.mPosition.data[0];

    viewport.posY = (((int)(viewHeight - config.mSize.data[1])) / (int) 2) + config.mPosition.data[1];

    viewport.sizeX = config.mSize.data[0];

    viewport.sizeY = config.mSize.data[1];

 

In the code above, as you can see, the viewport.posX and/or the viewport.posY may be negative, i.e. out of screen (this occurs whenever the config.mSize is larger than the viewWidth or the viewHeight); now, depending on the aspect ratio of the video config and the aspect ratio of the viewWidth, the viewport.posX negavtive value may become more obvious. This is what I was referring to when Imentioned the camera and screen aspect ratio in my previous post.

 

And please note that the problem only occurs on ios8.

OK, another reason for excessive cropping could be in the fact that iOS 8 handles the view width and view height as orientation-dependent (potentially breaking cmpatibility with code written for iOS 5 / 6 / 7);

You can check the iOS 8 migration guide for this (which also provides details and code snippets to adjust width/height handling on iOS 8, with a concrete example referring to the sample code in the Vuforia "Books" sample).

https://developer.vuforia.com/forum/ios/ios-8-xcode-6-and-vuforia-30-compatibility

 

 

Camera cropped with version 3

October 21, 2014 - 2:00am #4

Hi Alessandro,

 

Thanks for your reply.

On my screenshots, you can see the full camera feed as returned by the vuforia library.

There is clearly a difference between the display of the camera feed between the device photo app (even in HDR) and the feed returned by the library.

 

I dont understand your mention of the screen ratio, since this is handled in the function

- (void)configureVideoBackgroundWithViewWidth:(float)viewWidth andHeight:(float)viewHeight

Why would you need to crop anything outside it ?

 

 

By the way, I have exactly the same border loss in speed mode, plus a display ratio problem.

And please note that the problem only occurs on ios8.

 

Regards,

Camera cropped with version 3

October 20, 2014 - 11:02pm #3

W.r.t. 2.x,  the SDK 3.0 selects a higher camera resolution (i.e. for better image quality) on high-end devices, as mentioned here:

https://developer.vuforia.com/resources/dev-guide/hd-camera-view

Since HD resolutions tend to have wider aspect ratios (for example, 1280 x 720 => aspect ratio = 1.77, whereas a more conervative 640x480 has an aspect ratio of 1.33) this can result in an additional cropping effect; this is because in order to fit the camera image aspect ratio (1.77 for example) into the screen aspect ratio (1.33 for example) and still filling the entire screen, the camera image needs to be scaled up (otherwise you would see black borders, or alternatively you would have to stretch the camera image, i.e. violating its original aspect ratio).   

One way around this is to choose camera mode MODE_OPTIMIZE_SPEED, using the selectVideo mode method of the CameraDevice class, during camera initialization:

https://developer.vuforia.com/resources/api/class_q_c_a_r_1_1_camera_device

That mode will give you similar results to the ones you were seeing in SDK 2.x

Camera cropped with version 3

October 20, 2014 - 2:19am #2

I do not have access to Vuforia samples 2.8, but the samples v 3.0 also have this crop.

Do you have any news about this ?

Log in or register to post comments