OpenGL ES | Vuforia EDP

I am newbie on Vuforia. I am making an android application and want to change teapot for image target. I read many post about "Replacement of Teapot" but I understood it partically. I found some free 3D models(.obj) and opened it with notepad to get vertex arrays of model. There are many float numbers in mix form. I know that, I should change the vertex of teapot.java with new one. But I do not know how I do. I am not using NDK. How can I do it? And am I on right way?

Read more about How can I obtain vertex array of 3D Molde (.obj)?

Replace teapot with banana

By yazhi on Mon, 02/05/2018 - 05:41

Hi,

I managed to setup the default examples for image target, and I am able to see the teapot.

I am trying to replace the teapot object with the banana object for image targets, but can't see the banana. Don't see any console errors either.

I have gone through most of the related posts in this forums and other places, but couldn't figure out why.

Here are what I have changed in code:

- Updated the scale factor

// const float kObjectScaleNormal = 0.003f; const float kObjectScaleNormal = 100.f;

- Updated the render function renderFrameWithState:(const Vuforia::State&) state projectMatrix:(Vuforia::Matrix44F&) projectionMatrix

// glVertexAttribPointer(vertexHandle, 3, GL_FLOAT, GL_FALSE, 0, (const GLvoid*)teapotVertices); // glVertexAttribPointer(normalHandle, 3, GL_FLOAT, GL_FALSE, 0, (const GLvoid*)teapotNormals); // glVertexAttribPointer(textureCoordHandle, 2, GL_FLOAT, GL_FALSE, 0, (const GLvoid*)teapotTexCoords); glVertexAttribPointer(vertexHandle, 3, GL_FLOAT, GL_FALSE, 0, (const GLvoid*)bananaVerts); glVertexAttribPointer(normalHandle, 3, GL_FLOAT, GL_FALSE, 0, (const GLvoid*)bananaNormals); glVertexAttribPointer(textureCoordHandle, 2, GL_FLOAT, GL_FALSE, 0, (const GLvoid*)bananaTexCoords);

- and

// glDrawElements(GL_TRIANGLES, NUM_TEAPOT_OBJECT_INDEX, GL_UNSIGNED_SHORT, (const GLvoid*)teapotIndices); glDrawArrays(GL_TRIANGLES, 0, bananaNumVerts);

I am running VuforiaSamples-7-0-47 on iOS.

Any help is appreciated!

Read more about Replace teapot with banana

3D Object rendering

By Developercyg on Fri, 11/03/2017 - 18:30

Hello , Im new into 3D environment, so anyone can tell me how to render an obj file with multiple textures. I can only render an object with single texture, I dont know how to add multiple textures and load mtl files in Vuforia.. Please some one help me.. Thanks in advance

Read more about 3D Object rendering

Any plans to add number recognition?

By Strasza on Tue, 09/26/2017 - 18:16

Hi I am wondering if you have any plans to add number recognition to your unity asset?

The text recognition works very well but I would really need to be aböe to recognize numbers as well!

/Daniel

Read more about Any plans to add number recognition?

Replacing target shape with 2D Image

By Sefat on Tue, 07/25/2017 - 05:05

It has been achieved to replace the teapot with 2D image, but if I want to track the co ordinates of the target shape, what should I use? If I use this code below from "How To Draw a 2D Image on top of a Target using OpenGL ES" Vuforia::Vec2F targetSize = ((Vuforia::ImageTarget *) trackable)->getSize(); It causes error in Android Studio saying "Image Target is abstract function" What to do now to get the target coordinates and?

i don't understand what you

i don't understand what you mean but maybe you look at the VuMark Samples?

for the size something like this

opengles 2.0 error 1282 after call glDrawElements(…) method

By Strasza on Tue, 07/18/2017 - 15:36

I have some troubles with opengles 2.0. Currently I trying to display 3D teapot model without success. Method GLES20.glDrawElements generate opengl error with number 1282 and nothing is displayed on screen.

My draw method code:

public void draw(float[] tfMVPMatrix) {

GLES20.glUseProgram(miProgramID); miVertexPositionHandle = GLES20.glGetAttribLocation(miProgramID, VertexShader.Variables.VERTEX_POSITION.getName()); miVertexTexCoordsHandle = GLES20.glGetAttribLocation(miProgramID, VertexShader.Variables.VERTEX_TEXTURE_CORDS.getName()); miVertexNormalsHandle = GLES20.glGetAttribLocation(miProgramID, VertexShader.Variables.VERTEX_NORMALS.getName()); miProjectionMatrixHandle = GLES20.glGetUniformLocation(miProgramID, VertexShader.Variables.MVP_MATRIX.getName()); miColorHandle = GLES20.glGetUniformLocation(miProgramID, FragmentShader.Variables.COLOR.getName()); GLES20.glEnableVertexAttribArray(miVertexPositionHandle); GLES20.glVertexAttribPointer(miProgramID, 3, GLES20.GL_FLOAT, false, 3 * Float.SIZE, moVertBuff);

GLES20.glEnableVertexAttribArray(miVertexTexCoordsHandle); GLES20.glVertexAttribPointer(miProgramID, 2, GLES20.GL_FLOAT, false, 2 * Float.SIZE, moTexCoordBuff);

GLES20.glEnableVertexAttribArray(miVertexNormalsHandle); GLES20.glVertexAttribPointer(miProgramID, 3, GLES20.GL_FLOAT, false, 3 * Float.SIZE, moNormBuff);

GLES20.glUniform4fv(miColorHandle, 1, COLOR, 0); GLES20.glUniformMatrix4fv(miProjectionMatrixHandle, 1, false, tfMVPMatrix, 0);

GLES20.glDrawElements(GLES20.GL_TRIANGLES, miIndicesNumber, GLES20.GL_UNSIGNED_SHORT, moIndBuff); // opengl error 1282

GLES20.glDisableVertexAttribArray(miVertexPositionHandle); GLES20.glDisableVertexAttribArray(miVertexTexCoordsHandle); GLES20.glDisableVertexAttribArray(miVertexNormalsHandle); }

Vertex shader code:

attribute vec4 vPosition; attribute vec4 vNormal; attribute vec2 vTexture; varying vec2 texCoord; varying vec4 normal; uniform mat4 uMVPMatrix; void main() { gl_Position = uMVPMatrix * vPosition; normal = vNormal; texCoord = vTexture; }

Fragment shader code:

precision mediump float; varying vec4 normal; uniform vec4 vColor; void main() { gl_FragColor = vColor; }

My renderer code:

public class MyRenderer implements GLSurfaceView.Renderer { private int GLProgramID; private Demo3DObj oDemoTeapot; private final float[] mMVPMatrix = new float[16]; private final float[] mProjectionMatrix = new float[16]; private final float[] mViewMatrix = new float[16]; private float fRatio;

@Override public void onSurfaceCreated(GL10 gl, EGLConfig config) { GLProgram oProgram = new GLProgram(); GLProgramID = oProgram.build(); oDemoTeapot = new Demo3DObj(GLProgramID); GLES20.glClearColor(1.0f, 1.0f, 1.0f, 1.0f); }

@Override public void onSurfaceChanged(GL10 gl, int width, int height) { GLES20.glViewport(0, 0, width, height); fRatio = (float)width/height; Matrix.frustumM(mProjectionMatrix, 0, -fRatio, fRatio, -1, 1, 3, 7); }

@Override public void onDrawFrame(GL10 gl) { Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -3, 0f, 0f, 0f, 0f, 1.0f, 0.0f); Matrix.multiplyMM(mMVPMatrix, 0, mProjectionMatrix, 0, mViewMatrix, 0); GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT); oDemoTeapot.draw(mMVPMatrix); } }

Here is method, which is used to create moIndBuff:

private Buffer fillBufferI(int[] oContainer) { ByteBuffer bb = ByteBuffer.allocateDirect(2 * oContainer.length); bb.order(ByteOrder.LITTLE_ENDIAN); for (int s : oContainer) bb.putInt(s); bb.rewind(); return bb; }

I'm new in opengles world and I do not have any idea what can be wrong here. Can anybody help me solve this problem? What should I change here to display my model?

Read more about opengles 2.0 error 1282 after call glDrawElements(…) method

Digits recognize

By providar on Wed, 04/26/2017 - 14:18

Hello. Can i recognize float numbers with vuforia?

Hello,

Vuforia's Text Recognition feature supports the following characters: https://library.vuforia.com/articles/Solution/Characters-Supported-by-Text-Recognition

How to Render Text

By medabit on Wed, 04/05/2017 - 21:06

Hello,

I'm developing and Android AR application and I'd like to render text once a target is detected in order to display some informations about this target. I found these two links that demonstrate how to create a font and how to inject it to an openGL mechanism with the Android structure: - http://fractiousg.blogspot.fr/2012/04/rendering-text-in-opengl-on-android.html - https://github.com/d3alek/Texample2

Someone on another topic said that he was able to integrate this to the Vuforia SDK and its rendering system, without providing the mean to achieve it.

So I tried to adapt this project to fit with my own Vuforia project but, as I am an OpenGL beginner, I still can't find the proper solution to do what I want.

I'm sure that an experienced developer with Vuforia / OpenGL could find a solution to this problem with a quick look at my code.

Here is my ImageTargetRenderer.java file content:

import android.content.Context; import android.opengl.GLES20; import android.opengl.GLSurfaceView; import android.opengl.Matrix; import android.util.Log;

import com.soprasteria.arstore.business.TextRendering.GLText; import com.vuforia.Device; import com.vuforia.Matrix44F; import com.vuforia.State; import com.vuforia.Tool; import com.vuforia.Trackable; import com.vuforia.TrackableResult; import com.vuforia.Vuforia; import com.soprasteria.arstore.vuforia.AppRenderer; import com.soprasteria.arstore.vuforia.AppRendererControl; import com.soprasteria.arstore.vuforia.ApplicationSession; import com.soprasteria.arstore.vuforia.utils.CubeShaders; import com.soprasteria.arstore.vuforia.utils.LoadingDialogHandler; import com.soprasteria.arstore.vuforia.utils.Application3DModel; import com.soprasteria.arstore.vuforia.utils.Utils; import com.soprasteria.arstore.vuforia.utils.Teapot; import com.soprasteria.arstore.vuforia.utils.Texture;

import java.io.IOException; import java.util.Vector;

import javax.microedition.khronos.egl.EGLConfig; import javax.microedition.khronos.opengles.GL10;

// The renderer class for the ImageTargets public class ImageTargetRenderer implements GLSurfaceView.Renderer, AppRendererControl { private static final String LOGTAG = "ImageTargetRenderer";

private ApplicationSession applicationSession; private ImageTargets mActivity; private AppRenderer mAppRenderer;

private Vector<Texture> mTextures;

private int shaderProgramID; private int vertexHandle; private int textureCoordHandle; private int mvpMatrixHandle; private int texSampler2DHandle;

private Teapot mTeapot; //private FlyingSaucer mTeapot;

private float kBuildingScale = 0.012f; private Application3DModel mBuildingsModel;

private boolean mIsActive = false; private boolean mModelIsLoaded = false;

private static final float OBJECT_SCALE_FLOAT = 0.003f;

// TODO : Text Rendering test private GLText glText; // A GLText Instance private Context context; // Context (from Activity)

private int width = 100; // Updated to the Current Width + Height in onSurfaceChanged() private int height = 100; private float[] mProjMatrix = new float[16]; private float[] mVMatrix = new float[16]; private float[] mVPMatrix = new float[16]; //TODO fin

public ImageTargetRenderer(ImageTargets activity, ApplicationSession session) { mActivity = activity; applicationSession = session; // AppRenderer used to encapsulate the use of RenderingPrimitives setting // the device mode AR/VR and stereo mode mAppRenderer = new AppRenderer(this, mActivity, Device.MODE.MODE_AR, false, 0.01f , 5f);

// TODO context = mActivity.getApplicationContext(); }

// Called to draw the current frame. @Override public void onDrawFrame(GL10 gl) { if (!mIsActive) return;

// Call our function to render content from AppRenderer class mAppRenderer.render(); }

public void setActive(boolean active) { mIsActive = active;

if(mIsActive) mAppRenderer.configureVideoBackground(); }

// Called when the surface is created or recreated. @Override public void onSurfaceCreated(GL10 gl, EGLConfig config) { Log.d(LOGTAG, "GLRenderer.onSurfaceCreated");

// Call Vuforia function to (re)initialize rendering after first use // or after OpenGL ES context was lost (e.g. after onPause/onResume): applicationSession.onSurfaceCreated();

mAppRenderer.onSurfaceCreated();

// TODO début // Create the GLText glText = new GLText(context.getAssets());

// Load the font from file (set size + padding), creates the texture // NOTE: after a successful call to this the font is ready for rendering! glText.load( "Roboto-Regular.ttf", 14, 2, 2 ); // Create Font (Height: 14 Pixels / X+Y Padding 2 Pixels)

// enable texture + alpha blending GLES20.glEnable(GLES20.GL_BLEND); GLES20.glBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE_MINUS_SRC_ALPHA); // TODO Fin }

// Called when the surface changed size. @Override public void onSurfaceChanged(GL10 gl, int width, int height) { Log.d(LOGTAG, "GLRenderer.onSurfaceChanged");

// TODO début GLES20.glViewport(0, 0, width, height); float ratio = (float) width / height;

// Take into account device orientation if (width > height) { Matrix.frustumM(mProjMatrix, 0, -ratio, ratio, -1, 1, 1, 10); } else { Matrix.frustumM(mProjMatrix, 0, -1, 1, -1/ratio, 1/ratio, 1, 10); }

// Save width and height this.width = width; // Save Current Width this.height = height; // Save Current Height

int useForOrtho = Math.min(width, height);

//TODO: Is this wrong? Matrix.orthoM(mVMatrix, 0, -useForOrtho/2, useForOrtho/2, -useForOrtho/2, useForOrtho/2, 0.1f, 100f);

// TODO FIN

// Call Vuforia function to handle render surface size changes: applicationSession.onSurfaceChanged(width, height);

// RenderingPrimitives to be updated when some rendering change is done mAppRenderer.onConfigurationChanged(mIsActive);

initRendering(); }

// Function for initializing the renderer. private void initRendering() { GLES20.glClearColor(0.0f, 0.0f, 0.0f, Vuforia.requiresAlpha() ? 0.0f : 1.0f);

for (Texture t : mTextures) { GLES20.glGenTextures(1, t.mTextureID, 0); GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, t.mTextureID[0]); GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR); GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, t.mWidth, t.mHeight, 0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, t.mData); }

shaderProgramID = Utils.createProgramFromShaderSrc( CubeShaders.CUBE_MESH_VERTEX_SHADER, CubeShaders.CUBE_MESH_FRAGMENT_SHADER);

vertexHandle = GLES20.glGetAttribLocation(shaderProgramID, "vertexPosition"); textureCoordHandle = GLES20.glGetAttribLocation(shaderProgramID, "vertexTexCoord"); mvpMatrixHandle = GLES20.glGetUniformLocation(shaderProgramID, "modelViewProjectionMatrix"); texSampler2DHandle = GLES20.glGetUniformLocation(shaderProgramID, "texSampler2D");

if(!mModelIsLoaded) { mTeapot = new Teapot(); //mTeapot = new FlyingSaucer();

try { mBuildingsModel = new Application3DModel(); mBuildingsModel.loadModel(mActivity.getResources().getAssets(), "ImageTargets/Buildings.txt"); mModelIsLoaded = true; } catch (IOException e) { Log.e(LOGTAG, "Unable to load buildings"); }

// Hide the Loading Dialog mActivity.loadingDialogHandler .sendEmptyMessage(LoadingDialogHandler.HIDE_LOADING_DIALOG); }

}

public void updateConfiguration() { mAppRenderer.onConfigurationChanged(mIsActive); }

// The render function called from AppRendering by using RenderingPrimitives views. // The state is owned by AppRenderer which is controlling it's lifecycle. // State should not be cached outside this method. public void renderFrame(State state, float[] projectionMatrix) { // Renders video background replacing Renderer.DrawVideoBackground() mAppRenderer.renderVideoBackground();

GLES20.glEnable(GLES20.GL_DEPTH_TEST);

// handle face culling, we need to detect if we are using reflection // to determine the direction of the culling GLES20.glEnable(GLES20.GL_CULL_FACE); GLES20.glCullFace(GLES20.GL_BACK);

// Did we find any trackables this frame? for (int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++) { TrackableResult result = state.getTrackableResult(tIdx); Trackable trackable = result.getTrackable(); printUserData(trackable); Matrix44F modelViewMatrix_Vuforia = Tool .convertPose2GLMatrix(result.getPose()); float[] modelViewMatrix = modelViewMatrix_Vuforia.getData();

int textureIndex = trackable.getName().equalsIgnoreCase("stones") ? 0 : 1; textureIndex = trackable.getName().equalsIgnoreCase("tarmac") ? 2 : textureIndex;

// deal with the modelview and projection matrices float[] modelViewProjection = new float[16];

if (!mActivity.isExtendedTrackingActive()) { Matrix.translateM(modelViewMatrix, 0, 0.0f, 0.0f, OBJECT_SCALE_FLOAT); Matrix.scaleM(modelViewMatrix, 0, OBJECT_SCALE_FLOAT, OBJECT_SCALE_FLOAT, OBJECT_SCALE_FLOAT); } else { Matrix.rotateM(modelViewMatrix, 0, 90.0f, 1.0f, 0, 0); Matrix.scaleM(modelViewMatrix, 0, kBuildingScale, kBuildingScale, kBuildingScale); } Matrix.multiplyMM(modelViewProjection, 0, projectionMatrix, 0, modelViewMatrix, 0);

// TODO début // TEST: render the entire font texture glText.drawTexture( width/2, height/2, mVPMatrix); // Draw the Entire Texture

// TEST: render some strings with the font glText.begin( 1.0f, 1.0f, 1.0f, 1.0f, mVPMatrix ); // Begin Text Rendering (Set Color WHITE) glText.drawC("Test String 3D!", 0f, 0f, 0f, 0, -30, 0); // glText.drawC( "Test String :)", 0, 0, 0 ); // Draw Test String glText.draw( "Diagonal 1", 40, 40, 40); // Draw Test String glText.draw( "Column 1", 100, 100, 90); // Draw Test String glText.end(); // End Text Rendering

glText.begin( 0.0f, 0.0f, 1.0f, 1.0f, mVPMatrix ); // Begin Text Rendering (Set Color BLUE) glText.draw( "More Lines...", 50, 200 ); // Draw Test String glText.draw( "The End.", 50, 200 + glText.getCharHeight(), 180); // Draw Test String glText.end(); // End Text Rendering // TODO Fin

// activate the shader program and bind the vertex/normal/tex coords GLES20.glUseProgram(shaderProgramID);

if (!mActivity.isExtendedTrackingActive()) { GLES20.glVertexAttribPointer(vertexHandle, 3, GLES20.GL_FLOAT, false, 0, mTeapot.getVertices()); GLES20.glVertexAttribPointer(textureCoordHandle, 2, GLES20.GL_FLOAT, false, 0, mTeapot.getTexCoords());

GLES20.glEnableVertexAttribArray(vertexHandle); GLES20.glEnableVertexAttribArray(textureCoordHandle);

// activate texture 0, bind it, and pass to shader GLES20.glActiveTexture(GLES20.GL_TEXTURE0); GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextures.get(textureIndex).mTextureID[0]); GLES20.glUniform1i(texSampler2DHandle, 0);

// pass the model view matrix to the shader GLES20.glUniformMatrix4fv(mvpMatrixHandle, 1, false, modelViewProjection, 0);

// finally draw the teapot GLES20.glDrawElements(GLES20.GL_TRIANGLES, mTeapot.getNumObjectIndex(), GLES20.GL_UNSIGNED_SHORT, mTeapot.getIndices());

// disable the enabled arrays GLES20.glDisableVertexAttribArray(vertexHandle); GLES20.glDisableVertexAttribArray(textureCoordHandle); } else { GLES20.glDisable(GLES20.GL_CULL_FACE); GLES20.glVertexAttribPointer(vertexHandle, 3, GLES20.GL_FLOAT, false, 0, mBuildingsModel.getVertices()); GLES20.glVertexAttribPointer(textureCoordHandle, 2, GLES20.GL_FLOAT, false, 0, mBuildingsModel.getTexCoords());

GLES20.glEnableVertexAttribArray(vertexHandle); GLES20.glEnableVertexAttribArray(textureCoordHandle);

GLES20.glActiveTexture(GLES20.GL_TEXTURE0); GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextures.get(3).mTextureID[0]); GLES20.glUniformMatrix4fv(mvpMatrixHandle, 1, false, modelViewProjection, 0); GLES20.glUniform1i(texSampler2DHandle, 0); GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, mBuildingsModel.getNumObjectVertex());

Utils.checkGLError("Renderer DrawBuildings"); }

Utils.checkGLError("Render Frame");

}

GLES20.glDisable(GLES20.GL_DEPTH_TEST); }

private void printUserData(Trackable trackable) { String userData = (String) trackable.getUserData(); Log.d(LOGTAG, "UserData:Retreived User Data \"" + userData + "\""); }

public void setTextures(Vector<Texture> textures) { mTextures = textures;

}

Read more about How to Render Text

Text Rendering

By medabit on Wed, 04/05/2017 - 21:06

Hello, I would like to be able to recognize a target and then to render dynamic text onto this target.

I found on another topic that these two links provide a way to do this using openGL :

- http://fractiousg.blogspot.fr/2012/04/rendering-text-in-opengl-on-android.html - https://github.com/d3alek/Texample2 (using openGL 2.0)

I tried different things with that sample projects in order to adapt it to the Vuforia Rendering class with openGL. Unfortunately, I'm currently not able to find the right solution. Could someone please help me on the injection mechanism with the existing Vuforia model ? I think this is an openGL question since it directly interacts with it. I'm developing with Android Studio, based on the sample project provided by Vuforia (ImageTarget, ObjectRecognition, etc...)

EDIT : Here is my ImageTargetRenderer.java file content, I guess there is something to fix and, as I'm an OpenGL beginner, I still don't find the solution. Maybe some experts could find quickly what is wrong.