Log in or register to post comments

Is it possible to render 3D objects with sound?

February 8, 2011 - 1:25am #1

Hi, there.

1)

I've tried to render 3D objects with sound.
I spent much time to do it but I can't.:mad::confused:
For example, the dog shows up with dog's bark.

2)

Is it possible to take a photo with augmented objects?
I modified 'Dominoes' example.

Please, help me.... I couldn't go to bed for several days. Thanks.

Is it possible to render 3D objects with sound?

December 4, 2013 - 11:05pm #29

Hi, I tried the code by Muhammadgyan and it works fine in virtual button . .

When using this sound code, the sound  just can play "6 second" of the sound . .
Is that possible to play more than 6 sec of sound? for example 30 second . . .

Thanks 

Is it possible to render 3D objects with sound?

November 22, 2013 - 9:03pm #28

Hi, I had tried to put sound in virtual button as the example given below.

I had no error but the app cannot be run.

This is my code in cpp:

if (strcmp(button.getName(), "red") == 0)
				{
					playSoundEffect(0);

					// Assumptions:
					textureIndex = 0;
					const Texture* const thisTexture0 = textures[textureIndex];
					// Scale 3D model
					QCAR::Matrix44F modelViewScaled = modelViewMatrix;
					SampleUtils::scalePoseMatrix(kTeapotScale, kTeapotScale, kTeapotScale,
												 &modelViewScaled.data[0]);

					QCAR::Matrix44F modelViewProjectionScaled;
					SampleUtils::multiplyMatrix(&projectionMatrix.data[0],
												&modelViewScaled.data[0],
												&modelViewProjectionScaled.data[0]);

					// Render 3D model
					glUseProgram(shaderProgramID);

					glVertexAttribPointer(vertexHandle, 3, GL_FLOAT, GL_FALSE, 0,
										  (const GLvoid*) &teapotVertices[0]);
					glVertexAttribPointer(normalHandle, 3, GL_FLOAT, GL_FALSE, 0,
										  (const GLvoid*) &teapotNormals[0]);
					glVertexAttribPointer(textureCoordHandle, 2, GL_FLOAT, GL_FALSE, 0,
										  (const GLvoid*) &teapotTexCoords[0]);

					glEnableVertexAttribArray(vertexHandle);
					glEnableVertexAttribArray(normalHandle);
					glEnableVertexAttribArray(textureCoordHandle);

					glActiveTexture(GL_TEXTURE0);
					glBindTexture(GL_TEXTURE_2D, thisTexture0->mTextureID);
					glUniformMatrix4fv(mvpMatrixHandle, 1, GL_FALSE,
									   (GLfloat*)&modelViewProjectionScaled.data[0] );
					glUniform1i(texSampler2DHandle, 0 /*GL_TEXTURE0*/);
					glDrawElements(GL_TRIANGLES, NUM_TEAPOT_OBJECT_INDEX, GL_UNSIGNED_SHORT,
								   (const GLvoid*) &teapotIndices[0]);
					break;
				}

 

Is it possible to render 3D objects with sound?

July 2, 2013 - 7:37am #27

My apologies.

Even so, this is probably down to you to implement, depending on how you have set everything up.  If your device supports 3D Audio, then most likely it may support OpenAL which has full support for 3D sound including listeners, sources etc.

Sorry I cannot help any further here.

 

Is it possible to render 3D objects with sound?

July 2, 2013 - 3:34am #26

hi NalinS thx for your replies,,

but i'm not use unity for my engine,, i use vuforia with eclipse,,
do you have a suggestion for me to clear this for me? please help me

Is it possible to render 3D objects with sound?

July 1, 2013 - 4:03am #25

This is more of  Unity question, so you would probably have more luck searching/posting the forums at the Unity web site.

N

Is it possible to render 3D objects with sound?

June 29, 2013 - 6:39am #24

hi i'm new bee in AR application... i try codes  from muhammadgyan in imagetarget.cpp,, and it works,,
and then i try this code in virtual button sample,,

in metode RendererFrame at VirtualsButton.cpp

  if (buttonResult->isPressed())
  {
    // Run through button name array to find texture index
       for (int j = 0; j < NUM_BUTTONS; ++j)
         {
           if (strcmp(button.getName(), virtualButtonColors[j]) == 0)
               {

                 PlayEffectSound(j); }

//code

the sound is come out,,,butits looping when i cover the button..

I want to voice out only once for one-time pressing,, could you help me to give some suggestion , what should i do?

#sorry for my bad english

 

 

 

 

 

Is it possible to render 3D objects with sound?

October 18, 2012 - 9:30pm #23

muhammadgyan wrote:

i try the code in dominoes.cpp and it works. try this : put this code in imagetargets.cpp :

JNIEnv* javaEnv; jobject javaObj; jclass javaClass; // some code here // void playSoundEffect(int soundIndex, float volume = 1.0f); // some code here // JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_initNativeCallback(JNIEnv* env, jobject obj) { // Store the java environment for later use // Note that this environment is only safe for use in this thread javaEnv = env; // Store the calling object for later use // Make a global reference to keep it valid beyond the scope of this function javaObj = env->NewGlobalRef(obj); // Store the class of the calling object for later use jclass objClass = env->GetObjectClass(obj); javaClass = (jclass) env->NewGlobalRef(objClass); } int lastTrackableId = -1; JNIEXPORT void JNICALL Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv* env, jobject obj) { // some code here // if (trackable->getId() != lastTrackableId) { playSoundEffect(0); } lastTrackableId = trackable->getId(); } // // // some code here // // // void playSoundEffect(int soundIndex, float volume) { // Sound playback is always handled by the Android SDK // Use the environment and class stored in initNativeCallback // to call a Java method that plays a sound jmethodID method = javaEnv->GetMethodID(javaClass, "playSound", "(IF)V"); javaEnv->CallVoidMethod(javaObj, method, soundIndex, volume); }

and put this code in imagetargets.java :

private SoundManager mSoundManager; //put this code in method InitApplicationAR mSoundManager = new SoundManager(getApplicationContext()); mRenderer.setSoundManager(mSoundManager);

the last, put this code in imagetargetstrenderer.java :

private SoundManager mSoundManager; public native void initNativeCallback(); //put this code in method onsurfacecreated initNativeCallback(); // some code here // public void playSound(int soundIndex, float volume) { mSoundManager.playSound(soundIndex, volume); } /** Setter for the sound manager. */ public void setSoundManager(SoundManager soundManager) { mSoundManager = soundManager; }

dont forget to copy file SoundManager.java from dominoes into your imagetargets folder

 

I'm beginner with native ! could you please share the code ImageTargets.cpp ? 

Re: Is it possible to render 3D objects with sound?

June 14, 2011 - 5:08am #22

Another question for the soundpool is, if there is a posibility to play the soundclips one after the other when there are more markers in the stream. At the moment 3 soundclips are played at the same time.

Stephan

Re: Is it possible to render 3D objects with sound?

June 14, 2011 - 3:21am #21

Hi
I use the the SoundpoolExample from the DominoProject to play sound when a marker is in the camera. The sound is only played when i had marker in the camera but not while a marker is in the camera. Is it possible while a marker is in the stream?

Stephan

Re: Is it possible to render 3D objects with sound?

June 9, 2011 - 5:58am #20

There's a Virtual Button getID() method, you can do something similar with that.

- Kim

Re: Is it possible to render 3D objects with sound?

June 9, 2011 - 2:36am #19
rassall_jubair wrote:

Hi, Thanks. Its now playing four different sound on four different marker.
Regards
Rassall

Now I am trying to implement sound in virutalbutton example. I want every single button has different sound to play. but facing problem here to implement as my modified virtualbutton is detect button to draw object.
The following code is for detect image and play sound in Imagetarget example.

if (!strcmp(trackable->getName(), "ber2"))
{
if (trackable->getId() != lastTrackableId) {

playSoundEffect(0);
}

lastTrackableId = trackable->getId();

..

}

But I dont understand what should be change here for Virtualbutton example as virtualbutton detect the only button not trackable and also not count lastTrackableId /getId. like:

if (strcmp(button->getName(), "blue") == 0)
{

}

Can anyone suggest me here to solve this problem...

thanks

Rassal

Re: Is it possible to render 3D objects with sound?

June 6, 2011 - 4:33am #18

Hi ppreuss!

There is no error when replacing the code "trackable->getId()" with "marker->getMarkerId()", but the application doesn´t work and is closed immediately after the sample splash screen. I used exactly the same code as "muhammadgyan" posted.

Re: Is it possible to render 3D objects with sound?

June 5, 2011 - 11:48am #17

Hi Steph,

In the FrameMarkers.cpp file you can see that we check the type of the trackable and then set the new marker type to a variable called marker. Try replacing the code "trackable->getId()" with "marker->getMarkerId()"

-Peter

Re: Is it possible to render 3D objects with sound?

June 5, 2011 - 11:38am #16

Hi Rassall,

You may want to try checking trackable->http://ar.qualcomm.at/node/2000530https://ar.qualcomm.com/arforums/showthread.php?t=530&highlight=Status for an example of using the getStatus() function.

Re: Is it possible to render 3D objects with sound?

June 5, 2011 - 6:24am #15

I tried muhammadgyans Code (POST2612) in the FrameMarkers-SampleProject and got errormessages: If i put this code in the FrameMarkers.cpp, i get the errormessage that "trackable" was not declared in this scope. Can anybody help me with this error?

Steph

int lastTrackableId = -1;

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv* env, jobject obj)
{
//
some code here
//

if (trackable->getId() != lastTrackableId) {

playSoundEffect(0);
}

lastTrackableId = trackable->getId();
}

Re: Is it possible to render 3D objects with sound?

June 5, 2011 - 3:12am #14

Hi, Thanks. Its now playing four different sound on four different marker. Now the problem is if marker is disappear from camera still sound clip continue play until finish. Is it possible to implement the method that sound will go pause state once marker disappear and will play again when marker appear again in screen.

Regards
Rassall

Re: Is it possible to render 3D objects with sound?

June 4, 2011 - 4:44pm #13

It looks like you are checking the state of the last detected trackable. With this code I think sound will only play when moving between your two trackables. If you want to play sound every time a trackable is detected, try removing or commenting out the code:

if (trackable->getId() != lastTrackableId) { }

Thanks,

-Peter

Re: Is it possible to render 3D objects with sound?

June 4, 2011 - 3:29am #12

Thanks Muhammadgyan, It works for me.

Basically I am using four marker to draw four object. now want to play four different sound when detect four different marker. Now the problem is every time sound play once after detect marker, so 2nd time when marker detect again that time no sound play.

if (!strcmp(trackable->getName(), "ber2"))
{
textureIndex = 0;
thisVertices = vertices;
thisTexCoords = texture;
thisIndices = indices;
if (trackable->getId() != lastTrackableId) {

playSoundEffect(0);
}

lastTrackableId = trackable->getId();

}
else if (!strcmp(trackable->getName(), "lalu"))
{
textureIndex = 1;
thisVertices = vertices;
thisTexCoords = texture;
thisIndices = indices;

if (trackable->getId() != lastTrackableId) {

playSoundEffect1(0);
}

lastTrackableId = trackable->getId();
}

Here, sound play one time when marker detect first time. but no sound play after marker detect 2nd time/3rd time. So i want to play sound once when detect the marker everytime. So is there any to do this here in this coding.

Thanks
Rassall

Re: Is it possible to render 3D objects with sound?

June 3, 2011 - 5:07am #11

i try the code in dominoes.cpp and it works. try this :

put this code in imagetargets.cpp :

JNIEnv* javaEnv;
jobject javaObj;
jclass javaClass;

//
some code here
//

void playSoundEffect(int soundIndex, float volume = 1.0f);

//
some code here
//

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_initNativeCallback(JNIEnv* env, jobject obj)
{
    // Store the java environment for later use
    // Note that this environment is only safe for use in this thread
    javaEnv = env;

    // Store the calling object for later use
    // Make a global reference to keep it valid beyond the scope of this function
    javaObj = env->NewGlobalRef(obj);

    // Store the class of the calling object for later use
    jclass objClass = env->GetObjectClass(obj);
    javaClass = (jclass) env->NewGlobalRef(objClass);
}

int lastTrackableId = -1;

JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv* env, jobject obj)
{
//
some code here
//

if (trackable->getId() != lastTrackableId) {
                	 
                    playSoundEffect(0);
                	 }
                	
                    lastTrackableId = trackable->getId();
                }

//
//
//
some code here
//
//
//

void
playSoundEffect(int soundIndex, float volume)
{
    // Sound playback is always handled by the Android SDK
    // Use the environment and class stored in initNativeCallback
    // to call a Java method that plays a sound
    jmethodID method = javaEnv->GetMethodID(javaClass, "playSound", "(IF)V");
    javaEnv->CallVoidMethod(javaObj, method, soundIndex, volume);
}

and put this code in imagetargets.java :


private SoundManager mSoundManager;

//put this code in method InitApplicationAR
mSoundManager = new SoundManager(getApplicationContext());
        mRenderer.setSoundManager(mSoundManager);

the last, put this code in imagetargetstrenderer.java :


private SoundManager mSoundManager;

 public native void initNativeCallback();

//put this code in method onsurfacecreated
 initNativeCallback();

//
some code here
//

public void playSound(int soundIndex, float volume)
    {
        mSoundManager.playSound(soundIndex, volume);
    }
    
    /** Setter for the sound manager. */
    public void setSoundManager(SoundManager soundManager)
    {
        mSoundManager = soundManager;
    }

dont forget to copy file SoundManager.java from dominoes into your imagetargets folder

Re: Is it possible to render 3D objects with sound?

June 3, 2011 - 2:49am #10
robertross wrote:
//JNI code

if(!strcmp(trackable->getName(), "nameOfTargetImage"))
 //if targetImage is detected
{
        //Call to java playAudio
}


//Java code

public void playAudio() {  
	try {
....

Here how I can call to java playAudio. What will be code here to call playaudio. I am not sure that this public void playAudio() java code should place in ImageTargetsRenderer.java. Please can you give details little more.

Anyway I already trying same domineous java sound code in imagetargets.java,ImageTargetsRenderer.java, SoundManager.java and compling also no problem. but now cofused in imagetargets.cpp. How i can call java playAudio from here. Domineous.cpp use follwoing code to call. but i am not sure how to call this in imagetargets.cpp underfrom which jni class.

void
playSoundEffect(int soundIndex, float volume)
{
jmethodID method = javaEnv->GetMethodID(javaClass, "playSound", "(IF)V");
javaEnv->CallVoidMethod(javaObj, method, soundIndex, volume);
}

Can anyone help me to make it clear. or anyother way to play audio in imagetarget example.

Thanks
Rassall

Re: Is it possible to render 3D objects with sound?

June 2, 2011 - 11:29am #9

1) Did you copy the domino_tap sound file from the Dominoes project?

2) You're setting up the sound pool in the ImageTargets class, are you passing it to the ImageTargetsRenderer class in some way?

3) Try some debugging, is the playSound method ever called? You can add a breakpoint in Eclipse and run the debugger (for Java code at least).

- Kim

Re: Is it possible to render 3D objects with sound?

June 2, 2011 - 4:56am #8

i write the code :

mSoundPool = new SoundPool(4, AudioManager.STREAM_MUSIC, 0);
        mSoundIds = new ArrayList<Integer>();
        
        mSoundIds.add(mSoundPool.load(this, R.raw.domino_tap, 1));

into imagetargets.java ( inside method initApplication())

and the code :

 public void playSound(int soundIndex, float volume)
    {
    	    
        if (soundIndex < 0 || soundIndex >= mSoundIds.size()) {
            DebugLog.LOGE("Sound id " + soundIndex + " out of range");
            return;
        }
        
        mSoundPool.play(mSoundIds.get(soundIndex), volume, volume, 1, 0, 1.0f);
    }

into imagetargetsrenderer.java

it compile successfully, but when i starts the application, the sound didn't played..am i have something wrong??

Re: Is it possible to render 3D objects with sound?

June 2, 2011 - 1:58am #7

Instead of "context" can you use "this" or "this.getContext()"?

If you copy and paste the full error (or logcat) we could see whats going on.

Re: Is it possible to render 3D objects with sound?

June 1, 2011 - 9:15pm #6

i want sound played when trackable detected

i did this in ImageTargets.cpp

if (trackable->getId() != lastTrackableId) {
                	jclass javaClass = env->GetObjectClass(obj);
                	jmethodID method = env->GetMethodID(javaClass, "playSound", "(IF)V");
                	env->CallVoidMethod(obj, method, 0, 1.0f);
                    lastTrackableId = trackable->getId();
                }

and i did this in ImageTtargetsRrenderer.java

 // SoundPool is used for short sound effects
    private SoundPool mSoundPool;
    
    // An array of sound clip ids
    public ArrayList<Integer> mSoundIds;
.
.
.
.
public void playSound(int soundIndex, float volume)
    {
    	mSoundPool = new SoundPool(4, AudioManager.STREAM_MUSIC, 0);
        mSoundIds = new ArrayList<Integer>();
        
        mSoundIds.add(mSoundPool.load(context, R.raw.domino_tap, 1));
        
        if (soundIndex < 0 || soundIndex >= mSoundIds.size()) {
            DebugLog.LOGE("Sound id " + soundIndex + " out of range");
            return;
        }
        
        mSoundPool.play(mSoundIds.get(soundIndex), volume, volume, 1, 0, 1.0f);
    }

but i get an error for the "context"..can you resolve my problem and complete my code??

Re: Is it possible to render 3D objects with sound?

May 4, 2011 - 4:30am #5

I did something like this in native code:

int beepFlag = 1;

if (state.getNumActiveTrackables() > 0 && beepFlag == 1)

        {

            javaClass = env->GetObjectClass(obj);

            jmethodID playBeepMethod = env->GetMethodID(javaClass, "playBeep", "()V");

            env->CallVoidMethod(obj, playBeepMethod);

beepFlag = 0;
        }
else {
beepFlag = 1;
}

Re: Is it possible to render 3D objects with sound?

May 3, 2011 - 11:33pm #4

Hello, thanks to you now I'm able to play a sound when a target image is detected, but the sound is played a lot of times instead of just one.

To fix this I created a boolean variable called alreadyPlayed so when it's false the sound is played and the alreadyPlayed variable is changed to true to prevent the sound to be played again (this is working great). But now I don't know in which part of the code I have to reset the alreadyPlayed variable back to false (I want to put that part of the code where the target image is lost, but I don't know where this is)

Can anyone help me???

How can I play a sound when the target image is detected only (not when it's being tracked)

Re: Is it possible to render 3D objects with sound?

February 8, 2011 - 5:44am #3

1) You play your sounds in Java, as robertross has shown in his code snippets. The Dominoes application also has an example of playing short sound effects, that should get you started with the required JNI calls.

2) Taking a screen shot is going to be tricky, as Android doesn't have the proper hooks for creating a screen shot that includes OpenGL content (as far as I know). You might try using glReadPixels to get the OpenGL pixel buffer, and passing this up to Java where you can create a bitmap using it.

- Kim

Re: Is it possible to render 3D objects with sound?

February 8, 2011 - 3:16am #2
//JNI code

if(!strcmp(trackable->getName(), "nameOfTargetImage"))
 //if targetImage is detected
{
        //Call to java playAudio
}


//Java code

public void playAudio() {  
	try {
		MediaPlayer mMediaPlayer = MediaPlayer.create(this, R.raw.button37);  
		mMediaPlayer.setLooping(false);  
		Log.e("beep","started0");  
		mMediaPlayer.start(); 
		mMediaPlayer.setOnCompletionListener(new OnCompletionListener() {  
			public void onCompletion(MediaPlayer arg0) {  
				finish();  
			} 
		});
	} catch (Exception e) {  
		Log.e("beep", "error: " + e.getMessage(), e);  
	}  
}

Log in or register to post comments