Log in or register to post comments

glReadPixel() doens't taking screenshot with rendering 3D content

April 11, 2014 - 12:25am #1

I'm implementing the code for taking screenshot with live camera feed and 3D content. But when i click the button for taking screenshot it will takes only live camera feed it doesn't take 3D content what displaying on cameraview overlay. I'm using iOS 6.1. I used below code in ImageTargetEAGLView.

ImageTargetEAGLView.h:

 

 @interface ImageTargetEAGLView : UIView <UIGLViewProtocol>{

UIButton *photo;

UIImage *outputImage;

}

 }

 @property (nonatomic) BOOL takePhotoFlag;

 

 

ImageTargetEAGLView.mm:

 

  if (self) {

 takePhotoFlag = NO;

 [self DisplayPhotoButton];

}

return self;

 }

 

 - (void)DisplayPhotoButton

{

UIImage *closeButtonImage = [UIImage imageNamed:@"photo.png"];

 CGRect aRect = CGRectMake(20,20,

closeButtonImage.size.width,

closeButtonImage.size.height);

photo = [UIButton buttonWithType:UIButtonTypeCustom];

 photo.frame = aRect;

photo.userInteractionEnabled=YES;

[photo setImage:closeButtonImage forState:UIControlStateNormal];

[photo addTarget:self action:@selector(takePhoto) forControlEvents:UIControlEventTouchUpInside];

 [self  addSubview:photo];

 

}

 

 - (void)takePhoto

 {

NSLog(@"button pressed");

takePhotoFlag = YES;

}

 

Code for taking screenshot with 3D content on overlay glReadPixel():

 

- (UIImage*) glToUIImage

 {

 CGFloat scale = [[UIScreen mainScreen] scale];

 // CGRect s = CGRectMake(0, 0, 320.0f * scale, 480.0f * scale);

CGRect  s = CGRectMake(0, 0, (768.0f) * scale, 1024.0f * scale);

uint8_t *buffer = (uint8_t *) malloc(s.size.width * s.size.height * 4);

 glReadPixels(0, 0, s.size.width, s.size.height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);

CGDataProviderRef ref = CGDataProviderCreateWithData(NULL, buffer, s.size.width * s.size.height * 4, NULL);

CGImageRef iref = CGImageCreate(s.size.width, s.size.height, 8, 32, s.size.width * 4, CGColorSpaceCreateDeviceRGB(), kCGBitmapByteOrderDefault, ref, NULL, true, kCGRenderingIntentDefault);

size_t width = CGImageGetWidth(iref);

 size_t height = CGImageGetHeight(iref);

 size_t length = width * height * 4;

 uint32_t *pixels = (uint32_t *)malloc(length);

 CGContextRef context1 = CGBitmapContextCreate(pixels, width, height, 8, width * 4,

 CGImageGetColorSpace(iref), kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Big);

 

 CGAffineTransform transform = CGAffineTransformIdentity;

 transform = CGAffineTransformMakeTranslation(0.0f, height);

 transform = CGAffineTransformScale(transform, 1.0, -1.0);

 CGContextConcatCTM(context1, transform);

 CGContextDrawImage(context1, CGRectMake(0.0f, 0.0f, width, height), iref);

 CGImageRef outputRef = CGBitmapContextCreateImage(context1);

 

 outputImage = [UIImage imageWithCGImage: outputRef];

 

 CGDataProviderRelease(ref);

 CGImageRelease(iref);

 CGContextRelease(context1);

 CGImageRelease(outputRef);

 free(pixels);

 free(buffer);

 

UIImageWriteToSavedPhotosAlbum(outputImage, nil, nil, nil);

NSLog(@"Screenshot size: %d, %d", (int)[outputImage size].width, (int)[outputImage size].height);

return outputImage;

}

 

Below code will get the presentFrameBuffer from camera:

 

- (BOOL)presentFramebuffer

 {

if (takePhotoFlag)

 {

 NSLog(@"presentFrameBuffer");

 [self glToUIImage];

takePhotoFlag = NO;

}

// setFramebuffer must have been called before presentFramebuffer, therefore

 // we know the context is valid and has been set for this (render) thread

// Bind the colour render buffer and present it

 glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);

return [context presentRenderbuffer:GL_RENDERBUFFER];

 }

 

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 3:28am #23

This is my code:

- (BOOL)presentFramebuffer

{

     if (_takePhotoFlag)

      {

              [self glToUIImage];

              _takePhotoFlag = NO;

      }

    // setFramebuffer must have been called before presentFramebuffer, therefore

    // we know the context is valid and has been set for this (render) thread

    // Bind the colour render buffer and present it

    glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);

    return [context presentRenderbuffer:GL_RENDERBUFFER];

}

 

- (UIImage*) glToUIImage

 

{

UIImage *outputImage = nil;

        

        CGFloat scale = [[UIScreen mainScreen] scale];

        CGRect s = CGRectMake(0, 0, 320.0f * scale, 480.0f * scale);

        uint8_t *buffer = (uint8_t *) malloc(s.size.width * s.size.height * 4);

        

        glReadPixels(0, 0, s.size.width, s.size.height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);

        

        CGDataProviderRef ref = CGDataProviderCreateWithData(NULL, buffer, s.size.width * s.size.height * 4, NULL);

        

        CGImageRef iref = CGImageCreate(s.size.width, s.size.height, 8, 32, s.size.width * 4, CGColorSpaceCreateDeviceRGB(), kCGBitmapByteOrderDefault, ref, NULL, true, kCGRenderingIntentDefault);

        

        size_t width = CGImageGetWidth(iref);

        size_t height = CGImageGetHeight(iref);

        size_t length = width * height * 4;

        uint32_t *pixels = (uint32_t *)malloc(length);

        

        CGContextRef context1 = CGBitmapContextCreate(pixels, width, height, 8, width * 4,

                                                      CGImageGetColorSpace(iref), kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Big);

        

        CGAffineTransform transform = CGAffineTransformIdentity;

        transform = CGAffineTransformMakeTranslation(0.0f, height);

        transform = CGAffineTransformScale(transform, 1.0, -1.0);

        CGContextConcatCTM(context1, transform);

        CGContextDrawImage(context1, CGRectMake(0.0f, 0.0f, width, height), iref);

        CGImageRef outputRef = CGBitmapContextCreateImage(context1);

        

        outputImage = [UIImage imageWithCGImage: outputRef];

        

        CGDataProviderRelease(ref);

        CGImageRelease(iref);

        CGContextRelease(context1);

        CGImageRelease(outputRef);

        free(pixels);

        free(buffer);

         UIImageWriteToSavedPhotosAlbum(outputImage, nil, nil, nil);

        return outputImage;

          }

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 3:06am #22

Have you already looked at this link ?

https://developer.vuforia.com/forum/faq/android-how-can-i-capture-ar-view

This is written for Android, but the bulk of it is plain OpenGL code using glReadPixels(), which you can easily port to iOS;   now, depending on where exactly you put the glReadPixels() in your application code (e.g. after rendering the whole scene, or only part of it), the grabbed image will contain some of the  content (or all of it) appearing on the screen of your device.

The rest is app logic that you need to debug in your specific bit of code, as Nalin said.

 

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:49am #21

ok thanks. In while, if you have link please suggest me to refer.

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:46am #20

Sorry, we are not here to debug code.

I suggest you look at an alternative way of grabbing screenshots when the target moves out of view as maybe this might work.

 

N

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:41am #19

Can i send my code to you?

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:29am #18

the problem is if point the camera and overlay appera on ImageTarget, it taking screenshot. But if i move the camera from ImageTarget (see attached image url) and i'm getting overlay image, then the screenshot not saving with overlay.

https://developer.vuforia.com/sites/default/files/Screenshot%202014.04.11%2016.07.45.png

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:26am #17

Ah - ok.

In that case I am not sure why it is not working. You already have the code I posted up for native screenshots here:

https://developer.vuforia.com/forum/ios/screenshot-wont-taking-3d-content

 

...so not really sure what your next steps are here and why it is not working for you.

 

N

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:22am #16

I'm not using. It's native iOS code Books app

glReadPixel() doens't taking screenshot with rendering 3D conten

May 30, 2014 - 2:20am #15

The debug output means nothing to me.

The recommended way of grabing screenshots in Unity with and without augmentation is as I posted in this thread in August 2013

 

https://developer.vuforia.com/forum/unity-3-extension-technical-discussion/obtain-unity-camera-image-without-augmentation

 

HTH

N

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 4:17am #14

@Nalin: After debugging i got bellow things. Please have a look this:

 

2014-04-11 16:45:22.355 NaniAgroFoods[11881:907] 0

2014-04-11 16:45:22.357 NaniAgroFoods[11881:907] User tapped a link.

2014-04-11 16:45:22.434 NaniAgroFoods[11881:907] debug: shaderName=(Simple), shaderFileTYpe=(vertsh)

2014-04-11 16:45:22.436 NaniAgroFoods[11881:907] debug: shaderPath=(/var/mobile/Applications/C86276C0-53D5-42E0-8122-CFDF7405653E/NaniAgroFoods.app/Simple.vertsh)

2014-04-11 16:45:22.439 NaniAgroFoods[11881:907] debug: shaderName=(Simple), shaderFileTYpe=(fragsh)

2014-04-11 16:45:22.441 NaniAgroFoods[11881:907] debug: shaderPath=(/var/mobile/Applications/C86276C0-53D5-42E0-8122-CFDF7405653E/NaniAgroFoods.app/Simple.fragsh)

2014-04-11 16:45:22.463 NaniAgroFoods[11881:907] ARVC: Rotating to Portrait

2014-04-11 16:45:22.515 NaniAgroFoods[11881:672f] INFO/AR(11881) 2014-04-11 16:45:22: QCAR SDK version 2.8.7

2014-04-11 16:45:23.465 NaniAgroFoods[11881:8407] waitUntilInitFinished Execution Time: 0.596720

2014-04-11 16:45:23.467 NaniAgroFoods[11881:8407] target finder initialized

2014-04-11 16:45:23.486 NaniAgroFoods[11881:8407] *** -[AVCaptureVideoDataOutput setVideoSettings:] - videoSettings dictionary contains one or more unsupported (ignored) keys: (

    Width,

    AVVideoScalingModeKey,

    Height

)

2014-04-11 16:45:23.491 NaniAgroFoods[11881:8407] DEBUG/AR(11881) UIView has CAEAGLLayer class

2014-04-11 16:45:23.492 NaniAgroFoods[11881:8407] DEBUG/AR(11881) UIView responds to selector renderFrameQCAR

2014-04-11 16:45:24.327 NaniAgroFoods[11881:8407] ARVC: Rotating to Portrait

2014-04-11 16:45:26.425 NaniAgroFoods[11881:6c23] INFO/AR(11881) 2014-04-11 16:45:26: Completed CloudReco transaction with ID '8c5492360c2d471e9da0cf4d61449e72'

2014-04-11 16:45:26.519 NaniAgroFoods[11881:1703] INFO/AR(11881) 2014-04-11 16:45:26: ImageTracker: Successfully created dataset

2014-04-11 16:45:26.579 NaniAgroFoods[11881:1703] INFO/AR(11881) 2014-04-11 16:45:26: Successfully created ImageTarget.

2014-04-11 16:45:26.583 NaniAgroFoods[11881:1703] Successfully created new trackable 'wallmart' with rating '5'.

2014-04-11 16:45:26.929 NaniAgroFoods[11881:907] #DEBUG Using NSJSONSerialization

2014-04-11 16:45:26.930 NaniAgroFoods[11881:907] book url is http://www.server.net/projects/AR/Vuforia/Cloud_Reco/sparkling.png

2014-04-11 16:45:27.059 NaniAgroFoods[11881:907] thumb url is http://www.server.net/projects/AR/Vuforia/Cloud_Reco/sparkling.png

2014-04-11 16:45:29.734 NaniAgroFoods[11881:907] button pressed

2014-04-11 16:45:29.754 NaniAgroFoods[11881:9b07] presentFrameBuffer

2014-04-11 16:45:30.364 NaniAgroFoods[11881:9b07] Screenshot size: 768, 1024

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:58am #13

Did you try this? Because after scanning the book the information comes with close it appear long without targetImage. Then not able to take picture. Will try to debug and post the answer here.

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:46am #12

Not sure.

As always, best to step through with the debugger.

N

 

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:45am #11

I think the problem is, it won't taking screenshot of TargetOverlayView?

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:41am #10

Sorry Books app also taking picture with overlay book while book or target Image should display otherwise it's not taking. See the attached image. If i move the camera from target image it won't taking picture with overlay. Why?

AttachmentSize
Image icon Screenshot 2014.04.11 16.07.45.png997.35 KB

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:36am #9

In vuforia Sample app cloud Reco also taking picture with 3D. But Books app doesn't. May be this code will execute only for 3D?

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:32am #8

I uesd takePhotoFlag = NO; while taking picture. I changed some time before but then i didn't notice this. Sorry this is issue.

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:28am #7

Sorry. It's working in ImageTargets now

...as if by magic.  

What did you do to make it work?

 

N

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:28am #6

Can you help this please. Books app not taking screenshot with render book image. This is the issue.

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 3:07am #5

Sorry. It's working in ImageTargets now. thanks. But same code not working in Books app. 

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 2:55am #4

I suggest checking and tracing through again just to be sure.

The other thing you could try is on iOS7 to see if that makes a difference.

Is either device jailbroken?

 

N

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 2:45am #3

I'm also don't know about this. Everyone posted the same code but it's not working for me. I'm using iphone5 ios 6.1 and iPadMini iOS 6.1. In both it's not working.

glReadPixel() doens't taking screenshot with rendering 3D conten

April 11, 2014 - 2:37am #2

Not really sure what to say.

When I implemented the code that I posted up here:

https://developer.vuforia.com/forum/ios/screenshot-wont-taking-3d-content

... I was able to take a screenshot without issue - see attached picture.

This was on an iPhone 5  iOS7.

 

N

AttachmentSize
Image icon IMG_0035.JPG178.91 KB
Log in or register to post comments