Log in or register to post comments

AR Project

December 18, 2011 - 8:04am #1

Hi, I'm new to Android AR project.
Would anyone guide me to be familiar with my first AR project? :D
I really want to develope an AR app like this myself:
http://www.youtube.com/watch?v=plgOQyIim4A

I'm a beginner so that I just want to play some drum sounds when I press the "virtual buttons".
I finish all the installation process, can run the sample program on my phone.
I have finished reading the dev guide, but I'm still feeling vague of how can I acctually start writting an apps, what should I do next, how can I integrate the C++ code part into my program?

Thank you,

AR Project

October 23, 2012 - 12:17am #31

Cool ! Thanks for sharing this code. very useful !

AR Project

October 22, 2012 - 1:14pm #30

For all you guys asking how to play audio when an image is recognised this is how i did it:

1. First create a new mediaplayer global variable as

 

MediaPlayer mp = new MediaPlayer();

2. As soon as you find the trackable verify it in the c++ code and call this method of java through c++
if(trackable->getName(), "nameoftrackable") == 0 )
        {               
                if (trackable->getId() != lastTrackableId )
{
 jstring js = env->NewStringUTF(trackable->getName());
            jclass javaClass = env->GetObjectClass(obj);
            jmethodID method = env->GetMethodID(javaClass, "displayMessage", "(Ljava/lang/String;)V");
            env->CallVoidMethod(obj, method, js);
            lastTrackableId = trackable->getId();}}
 
3.Add this to ImageTargetsRenderer.java
 public static Handler mainActivityHandler;
    
    // Called from native to display a message
    public void displayMessage(String text)
    {
        // We use a handler because this thread cannot change the UI
    Message message = new Message();
    message.obj = text;
        mainActivityHandler.sendMessage(message);
        DebugLog.LOGD("player start");
    }
 
4.In ImageTargets.java in the onresume() method
add a if loop and verify the name of the trackable with the trackable for which you want the audio to play
 
if(str.equals("nameoftrackable"))
{
 
 
     try {
      //InputStream ins = getResources().openRawResource(R.raw.audiofile);
      mp = MediaPlayer.create(getApplicationContext(), R.raw.audiofile);
      mp.setLooping(true);
      DebugLog.LOGD("audioplay");
     } catch (IllegalArgumentException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
     } catch (IllegalStateException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
     }
   
     mp.start();
     
   }
 
5. In ImageTargets.cpp check check every frame that the trackable is still there if it's not again call a java function in the onresume() method and stop the audio player.
(Note: The str i am verifying here is the string that is defined as
  jstring js = env->NewStringUTF(trackable->getName());    )
You can call it with the string  "stop" instead of the string "trackable->getName" as used above.
 
 
if(str.equals("stop")){
try{
                        if(mp.isPlaying())
                        {
                        DebugLog.LOGD("true");
                              mp.stop();
                              mp.reset();
                             
                                                    }
                        }catch (Exception e) {
 
                            e.printStackTrace();
                            // TODO: handle exception
                        } }
 
Hope this helps. If any problems please ask.
 

 

AR Project

October 18, 2012 - 9:20pm #29

tuna_kren wrote:

Hi myticmoon,

can you share your code about SoundManager class in java? can i learn more about your code?

thanks before myticmoon... :)

 

 

Yea please myticmoon I'm also trying for a long time to play audio when image is recognized ,, but is still a complete headache :S
can you share your code ? 

AR Project

August 5, 2012 - 9:45pm #28

Hi myticmoon,

can you share your code about SoundManager class in java? can i learn more about your code?

thanks before myticmoon... :)

Re: AR Project

March 6, 2012 - 7:45pm #27

Sorry Kim, I have just fixed it!
Still having trouble with putting another 3D objects over markers. I never tried Perl before! T.T

Re: AR Project

March 6, 2012 - 7:37pm #26

Are there any errors in the log? Did you add both activities to the manifest?

- Kim

Re: AR Project

March 6, 2012 - 7:03pm #25

Hi Kim, I can't set another activity rather FrameMarkers as main activity.
For example, In this part of code I set Beginning_Activity as my main activity and press "Next" button to go to FrameMarkers activity. It just doesn't work.

package com.qualcomm.QCARSamples.FrameMarkers;

import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.AdapterView;
import android.widget.AdapterView.OnItemSelectedListener;
import android.widget.ArrayAdapter;
import android.widget.Button;
import android.widget.Spinner;

public class Beginning_Activity extends Activity implements OnItemSelectedListener, OnClickListener {
	public static boolean chose = false;
	private String choice;
	protected void onCreate(Bundle savedInstanceState){
		super.onCreate(savedInstanceState);
		setContentView(R.layout.activity1);
		Spinner spinner = (Spinner)findViewById(R.id.spinner1);
		ArrayAdapter<CharSequence> adapter = ArrayAdapter.createFromResource(
	            this, R.array.instruments_list, android.R.layout.simple_spinner_item);
	    adapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
	    Button button1 = (Button)findViewById(R.id.button1);
	    button1.setOnClickListener(this);
		spinner.setAdapter(adapter);
	    spinner.setOnItemSelectedListener(this);
	    
	}
	@Override
	public void onItemSelected(AdapterView<?> parent, View view, int pos,
			long id) {
		this.choice = parent.getItemAtPosition(pos).toString();
		int musical_type =0;
		// TODO Auto-generated method stub
		

		
		//newIntent.putExtras(bundle);
		
	}
	@Override
	public void onNothingSelected(AdapterView<?> arg0) {
		// TODO Auto-generated method stub
		
	}
	@Override
	public void onClick(View view) {
		// TODO Auto-generated method stub
		Bundle bundle = new Bundle();
		bundle.putString("choice", choice);
		Intent newIntent = new Intent(getBaseContext(), FrameMarkers.class);
		startActivity(newIntent);
	}
}

Re: AR Project

February 7, 2012 - 9:00pm #24

http://ar.qualcomm.at/node/2000283

- Kim

Re: AR Project

February 7, 2012 - 5:07am #23

Hi Kim, it just works nicely. It's much easier to do compare to the first time.
Now I can use two sticks to play drum and I got a feeling that I's a real drum :D.
Thank you a lot.
Now what should I do to change the 3D-objects placed over the frame markers.

THanks a lottt!! :DD

Re: AR Project

February 1, 2012 - 8:22pm #22

Sure, you have all of the information you need to do that. Just keep track of which markers are visible each frame. When a marker goes from active to lost record the time. When it is found again compare the current time with the recorded lost time.

- Kim

Re: AR Project

February 1, 2012 - 12:14am #21

Hi Kim,
Thank for ur response!
Is there any way to get around that? can I introduce an timer to detect an event that the frame marker is currently on the screen but disappears for 0.5 sec ( or shorter duration ) ?

Re: AR Project

January 31, 2012 - 7:14pm #20

No, virtual buttons only work with Image Targets. Sorry!

- Kim

Re: AR Project

January 30, 2012 - 10:32pm #19

Hi Kim,

Now the program run smoothly, I intend to improve my program a bit but I have to ask you about the possibility first!
Can I use Frame Marker as a virtual button?

Re: AR Project

January 19, 2012 - 8:12pm #18

Don't initialize your sounds every time playDrumSound is called (that's what your earlier code seemed to be doing). Check the bounds of your sound array to make sure it has an element for the given id.

- Kim

Re: AR Project

January 19, 2012 - 1:01am #17

OOps, look like program can run. However, the program only plays drum sound when I press "Blue Button", for the others, the program stops unexpectedly.:confused::confused::confused::confused:

Re: AR Project

January 19, 2012 - 12:07am #16

I see, have just fixed that problem

Quote:

for (int j = 0; j {
if (strcmp(button->getName(), virtualButtonColors[j]) == 0)
{
if (button->isPressed())
{
if (!buttonPressed[j])
{
// do something in response to the button press here
jclass javaClass = env->GetObjectClass(obj);
jmethodID method = env->GetMethodID(javaClass, "playDrumSound","(I)V" );
env->CallVoidMethod(obj, method, j);
buttonPressed[j] = true;
}
}
else
{
buttonPressed[j] = false;
}
break;
}
//anything else should be placed at this
}
}

No more bug, no problem with my playDrumSound method. However, each time, I press a virtual button, the program crashes, anything wrong with this.

Re: AR Project

January 18, 2012 - 10:32pm #15

Variable declaration order matters in C/C++. If you're using the code snippet I provided earlier, make sure this line comes after the NUM_BUTTONS global variable is defined:

const int NUM_BUTTONS = 4;

bool buttonPressed[NUM_BUTTONS];

- Kim

Re: AR Project

January 18, 2012 - 8:35am #14

These are the errors log when I tried ndk-build after modifying the native code. Still can't find out the error so far. :(

Quote:

jni/VirtualButtons.cpp:76: error: 'NUM_BUTTONS' was not declared in this scope
jni/VirtualButtons.cpp: In function 'void Java_com_qualcomm_QCARSamples_VirtualB uttons_VirtualButtonsRenderer_renderFrame(JNIEnv*, _jobject*)':
jni/VirtualButtons.cpp:275: error: 'buttonPressed' was not declared in this scop e
jni/VirtualButtons.cpp:287: error: 'buttonPressed' was not declared in this scop e
jni/VirtualButtons.cpp:294: error: 'button' was not declared in this scope
jni/VirtualButtons.cpp:330: error: 'vbCounter' was not declared in this scope
jni/VirtualButtons.cpp:336: error: 'vbVertices' was not declared in this scope
jni/VirtualButtons.cpp:341: error: 'modelViewProjection' was not declared in thi s scope
jni/VirtualButtons.cpp:346: error: 'target' was not declared in this scope
jni/VirtualButtons.cpp:354: error: 'textureIndex' was not declared in this scope
jni/VirtualButtons.cpp:359: error: 'modelViewMatrix' was not declared in this sc ope
jni/VirtualButtons.cpp: At global scope:
jni/VirtualButtons.cpp:393: error: expected constructor, destructor, or type con version before '(' token
jni/VirtualButtons.cpp:395: error: expected constructor, destructor, or type con version before '(' token
jni/VirtualButtons.cpp:396: error: expected constructor, destructor, or type con version before '(' token
jni/VirtualButtons.cpp:397: error: expected constructor, destructor, or type con version before '(' token
jni/VirtualButtons.cpp:399: error: expected constructor, destructor, or type con version before '.' token
jni/VirtualButtons.cpp:643: error: expected declaration before '}' token
/cygdrive/c/Development/android-ndk-r7-windows/android-ndk-r7/build/core/build-b inary.mk:241: recipe for target `obj/local/armeabi/objs-debug/VirtualButtons/Vir tualButtons.o' failed

Re: AR Project

January 17, 2012 - 7:26pm #13

Hmm, why are you dereferencing the env pointer now? Try using "env" instead of "(*env)".

- Kim

Re: AR Project

January 17, 2012 - 8:18am #12

Hi Kim,
Could you check for the whether this code is correct or not?

Quote:

jclass javaClass = (*env)->GetObjectClass(obj); //changed from obj to jobject
jmethodID method = (*env)->GetMethodID(javaClass, "playDrumSound", "(I)V"); //method is playSound method in Java already
(*env)->CallVoidMethod(env, obj, method, j); //changed from obj to jobject

The method parameters:

Quote:

Java_com_qualcomm_QCARSamples_VirtualButtons_VirtualButtonsRenderer_renderFrame(JNIEnv *env, jobject obj)

Re: AR Project

January 16, 2012 - 11:20pm #11

Hi, I have just upgraded my phone to IceCream SandWich and recognized that Qualcomm AR doesn't support ICS. Even though I really enjoy ICS, I have to flash 2.3.4 ROM to my phone again. T.T

I use debugger on Eclipse I find out that the playSound method is never called by the native code. Is there any error with the code related to JNI? :(

Re: AR Project

January 15, 2012 - 9:40am #10

You need to add a variable name for those parameters, e.g.

Java_com_qualcomm_QCARSamples_VirtualButtons_Virtu alButtonsRenderer_renderFrame(JNIEnv* env, jobject obj)

- Kim

Re: AR Project

January 15, 2012 - 6:46am #9

Hi Kim,

I haven't found where is the errors yet.
Just a question, the parameter for this method should be "obj" or "jobject"

Quote:

jclass javaClass = env->GetObjectClass(obj);

since in this method

Quote:

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_VirtualButtons_VirtualButtonsRenderer_renderFrame(JNIEnv *, jobject)

the parameter is jobject

Re: AR Project

January 13, 2012 - 2:09pm #8

I just noticed that you're adding the playSound method to VirtualButtons.java instead of VirtualButtonsRenderer.java. The renderFrame method is called from VirtualButtonsRenderer, so when you do a JNI method lookup using that env it's looking for a method in the VirtualButtonsRenderer class.

Try moving the playSound method to the Renderer. Make sure it is getting called, either by adding a print statement or using the Eclipse debugger.

Also, it isn't a great idea to recreate the sound manager every time the method is called. You probably want to do that just once.

- Kim

Re: AR Project

January 12, 2012 - 9:55pm #7

I have added sound file to the res/raw already. However, it just doesn't work. I'm trying to find out where are the error(s).

Thank a lot!

Re: AR Project

January 11, 2012 - 7:02pm #6

The first thing I see is an incorrect method signature:

jmethodID method = env->GetMethodID(javaClass, "playSound", "(Ljava/lang/StringV");

Should be:

jmethodID method = env->GetMethodID(javaClass, "playSound", "(I)V");

(Your method takes an int rather than a String)

Add your sound files to the res/raw folder.

- Kim

Re: AR Project

January 11, 2012 - 7:26am #5

Hi Kim,
I have modified the code, but looks like there're some errors. This is the 1st time I dealing with Java Native Interface. So I'm not very sure I'm doing correctly or not.

Quote:

for (int i = 0; i getNumVirtualButtons(); ++i)
{
const QCAR::VirtualButton* button = target->getVirtualButton(i);

for (int j = 0; j {
if (strcmp(button->getName(), virtualButtonColors[j]) == 0)
{
if (button->isPressed())
{
if (!buttonPressed[j])
{
// do something in response to the button press here
//jstring js = env->NewStringUTF(trackable->getName());
jclass javaClass = env->GetObjectClass(obj);
jmethodID method = env->GetMethodID(javaClass, "playSound", "(Ljava/lang/String;)V"); //method is playSound method in Java already
env->CallVoidMethod(obj, method, j);
buttonPressed[j] = true;
}
}
else
{
buttonPressed[j] = false;
}
break;
}
}
}

In the VirtualButtons.java I add this method

Quote:

public void playSound(int index){
SoundManager mSoundManager = new SoundManager();
mSoundManager.initSounds(getBaseContext());
mSoundManager.addSound(1, R.raw.drum1); //how to add to R.raw argggggg
mSoundManager.addSound(2, R.raw.drum2);
mSoundManager.addSound(3, R.raw.drum3);
mSoundManager.addSound(4, R.raw.drum4);
switch(index){
case 1:
mSoundManager.playSound(1);
break;
case 2:
mSoundManager.playSound(2);
break;
case 3:
mSoundManager.playSound(3);
break;
case 4:
mSoundManager.playSound(4);
break;
}
}

Regards,

Re: AR Project

January 10, 2012 - 8:46pm #4

That was just some sample code to get you started, you'll have to adapt it to your project. Look in VirtualButtons.cpp instead of ImageTargets.cpp. The samples are similar in structure.

- Kim

Re: AR Project

January 10, 2012 - 6:35am #3

Hi Kim,

You said to add this code segment to ImageTargets.cpp, but in Virtual Buttons project. There is no ImageTargets.cpp.

int lastTrackableId = -1;

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv* env, jobject obj)
{
...

// Did we find any trackables this frame?
for(int tIdx = 0; tIdx {
// Get the trackable:
const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);

// Compare this trackable's id to a globally stored id
// If this is a new trackable, find the displayMessage java method and
// call it with the trackable's name
if (trackable->getId() != lastTrackableId) {
jstring js = env->NewStringUTF(trackable->getName());
jclass javaClass = env->GetObjectClass(obj);
jmethodID method = env->GetMethodID(javaClass, "displayMessage", "(Ljava/lang/String;)V");
env->CallVoidMethod(obj, method, js);
lastTrackableId = trackable->getId();
}

...
}

...
}

Re: AR Project

December 19, 2011 - 12:30pm #2

For your idea, I would recommend starting with the VirtualButtons sample. Take a look at the native renderFrame method (in VirtualButtons.cpp), at the code around button->http://ar.qualcomm.at/node/2000032getNumVirtualButtons(); ++i)
{
const QCAR::VirtualButton* button = target->getVirtualButton(i);

for (int j = 0; j {
if (strcmp(button->getName(), virtualButtonColors[j]) == 0)
{
if (button->isPressed())
{
if (!buttonPressed[j])
{
// do something in response to the button press here

buttonPressed[j] = true;
}
}
else
{
buttonPressed[j] = false;
}
break;
}
}
}
[/CODE]

Take a look at this post for some code that displays a message in Java in response to a tracking event in C++: https://ar.qualcomm.at/arforums/showthread.php?p=70&postcount=2

You may want to do something similar to play your sound in Java.

- Kim

Log in or register to post comments