Android – How can I access the camera image
This article explains how to extract one of the images provided by the QCAR::Frame object, using a desired image format, and how to convey the image data from native C++ to Java (Android).
Taking the Image Targets sample code as a reference, you can follow these steps to achieve what is described above:
- In ImageTargets.cpp, add the following two global variables (javaVM and activityObj) and the function JNI_OnLoad (this function is automatically invoked and is used here for initializing the javaVM object, which represents the Java Virtual Machine of our Android Activity):
//global variables
JavaVM* javaVM = 0;
jobject activityObj = 0;
JNIEXPORT jint JNICALL
JNI_OnLoad(JavaVM* vm, void* reserved) {
LOG("JNI_OnLoad");
javaVM = vm;
return JNI_VERSION_1_4;
}
- Initialize the activityObj object by adding this code at the end of the initApplicationNative function:
activityObj = env->NewGlobalRef(obj);
- Add the following code at the beginning of the QCAR_onUpdate function:
QCAR::Image *imageRGB565 = NULL;
QCAR::Frame frame = state.getFrame();
for (int i = 0; i < frame.getNumImages(); ++i) {
const QCAR::Image *image = frame.getImage(i);
if (image->getFormat() == QCAR::RGB565) {
imageRGB565 = (QCAR::Image*)image;
break;
}
}
if (imageRGB565) {
JNIEnv* env = 0;
if ((javaVM != 0) && (activityObj != 0) && (javaVM->GetEnv((void**)&env, JNI_VERSION_1_4) == JNI_OK)) {
const short* pixels = (const short*) imageRGB565->getPixels();
int width = imageRGB565->getWidth();
int height = imageRGB565->getHeight();
int numPixels = width * height;
jbyteArray pixelArray = env->NewByteArray(numPixels * 2);
env->SetByteArrayRegion(pixelArray, 0, numPixels * 2, (const jbyte*) pixels);
jclass javaClass = env->GetObjectClass(activityObj);
jmethodID method = env-> GetMethodID(javaClass, "processCameraImage", "([BII)V");
env->CallObjectMethod(activityObj, method, pixelArray, width, height);
env->DeleteLocalRef(pixelArray);
}
}
- Again in ImageTargets.cpp, in the startCamera function, add the code to specify the desired frame format, right after calling QCAR::CameraDevice::getInstance().start(), as shown in the following snippet:
// Start the camera:
if (!QCAR::CameraDevice::getInstance().start())
return;
QCAR::setFrameFormat(QCAR::RGB888, true);
// etc ...
- Finally, in ImageTargets.java, add a method called “processCameraImage” to process the byte array representing the image passed from JNI code; for instance, the following code uses the image buffer (byte[]) to build an Android Bitmap:
public void processCameraImage(byte[] buffer, int width, int height)
{
Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565);
bitmap.copyPixelsFromBuffer(ByteBuffer.wrap(buffer));
}