Log in or register to post comments

Overlay Android view with image target

May 17, 2017 - 9:34pm #1

Hello,

I am trying to overlay an Android view (such as Webview) with image target after the target is successfully recognized. I cannot draw the view with Texture, because my Android view needs to respond to user inputs (such as touch events, keyboard inputs, etc.). Here is the approach I am taking:

  1.  From the Vuforia OpenGL renderer, I can obtain the Pose of the TrackableResult, and then further obtain ModelView Matrix and Projection Matrix
  2. Apply the pose / modelview matrix / projection matrix to Android view via Animation, to achieve 3D rendering effect of the view, and retain correct handling of user input (such as touch events).

Problem is that Android Animation (such as via ViewPropertyAnimator) does not take any matrix. It has only methods like ViewPropertyAnimator.x().y().scaleX().scaleY().rotationX().rotationY().rotation(). 

I was having a very hard time to calculate animation parameters such as scaleX, scaleY, translations, and rotation angles from the modelview matrix and projection matrix. 

 

To simplify the problem, I created a sample code here: https://github.com/huahou/webviewsample

I also paste the code here:

import android.graphics.Color;

import android.os.Bundle;

import android.os.Handler;

import android.support.v7.app.AppCompatActivity;

import android.webkit.WebView;

import android.webkit.WebViewClient;

public class MainActivity extends AppCompatActivity {

    private WebView mWebview;

    @Override

    protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        setContentView(R.layout.activity_main);

        mWebview = (WebView)findViewById(R.id.webview);

        mWebview.setWebViewClient(new WebViewClient());

        mWebview.getSettings().setJavaScriptEnabled(true);

        mWebview.setBackgroundColor(Color.TRANSPARENT);

        mWebview.loadUrl("http://www.cnn.com");

        new Handler().postDelayed(new Runnable() {

            @Override

            public void run() {

                // The below modelview matrix and projection matrix data are obtained from openGL code

                float[] modelViewMatrixData = {-0.037363f, -0.883836f, 0.466301f, 0.000000f, -0.998281f, 0.011926f, -0.057385f, 0.000000f, 0.045158f, -0.467644f, -0.882763f, 0.000000f, 7.434665f, -4.837492f, 276.385986f, 1.000000f};

                float[] projectionMatrixData = {0.000000f, -1.649297f, 0.000000f, 0.000000f, -2.940194f, 0.000000f, 0.000000f, 0.000000f, 0.026083f, 0.011500f, 1.004008f, 1.000000f, 0.000000f, 0.000000f, -20.040081f, 0.000000f};

                // Help needed: How to calculate below parameters from modelview matrix and projection matrix above

                // so that webview overlays (pins to) the image drawn in openGL?

                // below are some dummy values to illustrated how I position the webview:

                float scaleX = 0.5f;

                float scaleY = 0.2f;

                float x = 10f;

                float y = 100f;

                float rotationZ = 5f;

                float rotationX = 0f;

                float rotationY = 25f;

                mWebview.animate()

                        .scaleX(scaleX)

                        .scaleY(scaleY)

                        .x(x)

                        .y(y)

                        .rotation(rotationZ)

                        .rotationX(rotationX)

                        .rotationY(rotationY);

            }

        }, 5000);

    }

}

Here is the activity_main.xml:

<?xml version="1.0" encoding="utf-8"?>

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"

    xmlns:app="http://schemas.android.com/apk/res-auto"

    xmlns:tools="http://schemas.android.com/tools"

    android:layout_width="match_parent"

    android:layout_height="match_parent"

    tools:context="com.mobisigma.webviewglsample.MainActivity">

    <ImageView

        android:layout_width="match_parent"

        android:layout_height="match_parent"

        android:scaleType="fitXY"

        android:src="@drawable/img"/>

    <WebView

        android:id="@+id/webview"

        android:layout_width="match_parent"

        android:layout_height="match_parent" />

</RelativeLayout>

 

Anybody has similar experience implementing such solution? If so, do you mind sharing some ideas?

Any inputs are greatly appreciated!

Thank you!

 

 

Log in or register to post comments