Log in or register to post comments

"Water (Pro Only)" Unity package reflection bug

January 30, 2012 - 7:34am #1

tl;dr This issue is fixed in Unity 3.5.2.

Hello,

I'm using QCAR iOS 1.5.3 beta1 & Unity 3.4.2f3 for iOS.
I created a new project, following the steps described in the 1.5 Beta documentation to get a working Image Target application. I used the Stones marker available from the qcar-unity-imagetargets-ios-1-5-3-beta1 Unity package.
I imported the Water (Pro Only) package, used a Daylight Water prefab instance to work as the 3D augmentation, and set the Water Mode to Reflective.
I added a stretched cube to test the water reflection, and ran the app.
On both iPod Touch and iPad 2, I get the same behavior. It feels like the reflection position is not updated after it is first displayed, like a buffer not being refreshed. After some time, an error is displayed each frame in Xcode's console:
"
OpenGLES error 0x0506 in /Unity/QCAR_Water/iOSbuild/Classes/iPhone_GlesSupport.cpp:204
"
I don't know what triggers it. This error is mentioned in this thread: Invalid OpenGL operation.

I don't know much about shading, and whether this error is related to QCAR or Unity. The problem can't be seen if running the application in the editor and moving around the scene.
I uploaded the test project on this page:
http://www.mediafire.com/?paeeza17zoi83i6.

Any help would be greatly appreciated!

Thanks.

"Water (Pro Only)" Unity package reflection bug

October 21, 2014 - 2:12am #28

Ok I have an update.  The good news is issue number 2 doesn't seem to apply when its run on the device, so as long as I scale up the plane and cut off abit of the camera footage I have a workable, if alittle amataur,  solution.  The bad news is I occationally run into OpenGLES error 0x0502 when depolying to the device from xcode.  This also used to happen whenever the device changed orientation before I tweaked my edited VideoTextureBehaviour so it nolonger calls UpdateBehaviour() on device rotation.  

"Water (Pro Only)" Unity package reflection bug

October 20, 2014 - 4:13am #27

Hello, sorry to post in such an old thread, feel free to tell me to start a new one, I just thought I should use this one as it provides clear context.

Basically I require some Unity water reflection in my latest AR project, however as everybodies nodoubt aware by now, getting round the reflection bug isn't easy.   Water4Example isn't the solution it appears to be at first glance, because its default settings do very little reflection of other scene objects, turn the reflection up on it and you run into the same bug as with pro only daylight water.  

I've been impimenting Kim's genius solution quoted below and have had some success.

ksiva wrote:

This part is rather challenging, especially since you don't know the FOV of the camera (QCAR sets this under the hood). One approach is to do some raycasting to find the bounds of the far clip plane and fit the mesh to that. Here's a snippet that might help:

// Scale game object for full screen video image: gameObject.transform.localScale = new Vector3(1, 1, 1 * (float)mTextureInfo.imageSize.y / (float)mTextureInfo.imageSize.x); // Position the mesh at the far end of the perspective frustum // Choose a point almost at the far clip plane float dist = m_Camera.farClipPlane * 0.99f; // Define a plane at the chosen distance Plane farPlane = new Plane(m_Camera.transform.forward, m_Camera.transform.position + m_Camera.transform.forward * dist); // Cast a ray along the lower left and upper right edges of the frustum Ray ray1 = m_Camera.ScreenPointToRay(new Vector3(0, 0, 0)); Ray ray2 = m_Camera.ScreenPointToRay(new Vector3(m_Camera.pixelWidth, m_Camera.pixelHeight, 0)); // Find the points at which the rays intersect the plane float rayDist = 0.0f; farPlane.Raycast(ray1, out rayDist); Vector3 p1 = m_Camera.transform.InverseTransformPoint(ray1.GetPoint(rayDist)); farPlane.Raycast(ray2, out rayDist); Vector3 p2 = m_Camera.transform.InverseTransformPoint(ray2.GetPoint(rayDist)); // Position the video mesh at the chosen distance gameObject.transform.localPosition = new Vector3(0, 0, dist); // Scale the video mesh to stretch between the intersection points gameObject.transform.localScale *= (p2.x - p1.x) / 2.0f;

- Kim

 

I currently have a plane rendering the camera feed appearing just before the farclipplane of the only functioning camera in the scene, which is a perspective ARCamera (with solid colour clear flags.)  However this only semi-works and is prevented from being a perfect solution by two issues:

1. The display ratio as dictated by mTextureInfo (QCARRenderer.VideoTextureInfo) seems to result in the plane being the appropriate width for the screen, but leaving gaps at the top and bottom of the screen, presumably because the camera is perspective not orthographic.  I can resize the plane to get around this but by doing this i'm stretching or obscuring parts of the camera texture which isn't a smart move. 

 

2. Regardless of if I resize the plane the AR, which renders perfectly at the centre of the screen, now drifts off target when the target is moved to the sides of the screen, which I assume is due to a disconnect between where the background is being rendered and where the AR positioner thinks its being rendered.

 

I'd be really grateful if anyone has a solution or any advice to getting any kind of reflective water or mirrorreflection working with vuforia AR, seems a little crazy if in two years this hasn't been solved... I mean, think of all the cool AR things people could be doing with glasses of water which react to being poured etc.

 

"Water (Pro Only)" Unity package reflection bug

July 11, 2012 - 2:20am #26

Good to hear - keep me posted

N

"Water (Pro Only)" Unity package reflection bug

July 10, 2012 - 7:46am #25

Thanks for the reply. We didn't try the clipping plane solution that was posted, but discovered that Material Water4Example (Advanced) worked well instead.

With nightime water, I do think it has something to do with reflections in the renderer, and it may not be picking up a skydome.

So for now we have a work around.

 

"Water (Pro Only)" Unity package reflection bug

July 10, 2012 - 5:55am #24

Hi Eric,

I'll see if we can look into this further.

Please PM me to progress.

 

N

"Water (Pro Only)" Unity package reflection bug

July 10, 2012 - 5:22am #23

Bump!

 

 

We still can not get this working. The Water (nightime water PRO, yes we have pro abd iOS pro ) renders black on our iPads. When we take out the QAR sections of the build it displays correctly. We are making an AR app that needs moving water and need a solution as soon as possible. Anyone have any suggestions? 

Kim, it looks like you had a solution, do you have any advice for us?

Anyone else find a work around? 

Thanks,

Eric

Re: "Water (Pro Only)" Unity package reflection bug

May 29, 2012 - 12:49am #22

This bug is fixed in Unity 3.5.2!

Re: "Water (Pro Only)" Unity package reflection bug

April 26, 2012 - 6:28am #21

The error from the stripped down sample is fixed for the next Unity release (the one after v3.5.1f2). Let's hope that it fixes the initial bug!

Re: "Water (Pro Only)" Unity package reflection bug

March 30, 2012 - 3:23am #20

Thanks for the quick answer.

New question asked on Unity Answers, let's see if it helps!

Re: "Water (Pro Only)" Unity package reflection bug

March 29, 2012 - 4:09pm #19

I don't see anything obvious, but then I'm not terribly familiar with Unity's RenderTexture class. I think it's great that you've narrowed it down to this bit of code, hopefully Unity will be able to pinpoint the issue.

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

March 29, 2012 - 7:37am #18

I've stripped down the sample as much as I could so that it would still trigger the OpenGL error. There's nothing related to Vuforia in the sample anymore but a single call to "GL.InvalidateState()".
I was wondering if you (Kim) could take a look at it in case you see something obvious before I ask on Unity answers and/or report another bug to the Unity team.

There are only two game objects left in the sample:

- a simple game object with a renderer component but no mesh filter. A script component creates a camera and a render texture, and manually renders the camera inside the render texture on "OnWillRenderObject". This game object represents the water part.

- a default main camera, with a script that calls GL.InvalidateState() on "Update". This camera represents the Vuforia part.

On iPad 2, the Gl error shows up each frame after a few seconds.
On iPod Touch 4, the Gl error does not show up but after a while, a memory warning is issued and the app is terminated.
(Tested with Unity 3.5, Vuforia iOS 1.5.9, Xcode 4.3.1, Mac OS X 10.7.3)

Thank you!

Re: "Water (Pro Only)" Unity package reflection bug

February 29, 2012 - 5:38am #17

Thank you for the suggestion.

Re: "Water (Pro Only)" Unity package reflection bug

February 28, 2012 - 7:05pm #16

You may want to try submitting the project via Unity's "Report a Bug" option in the help menu. You can attach files (like the target) and give a concise description to make it easy to reproduce.

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 28, 2012 - 9:01am #15

I've updated the sample with Vuforia 1.5 and Unity 3.5. I added some information to my initial question.

Unfortunately the bug is still there!

Re: "Water (Pro Only)" Unity package reflection bug

February 24, 2012 - 8:54am #14

Yes, check out the BackgroundTextureAccess sample in the new 1.5 release. There's a Readme included that describes how to render the video background using the ARCamera. Also, with this release your settings to the camera clear flags and background color in the ARCamera inspector are respected.

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 24, 2012 - 2:22am #13

[QUOTE=ksiva" />http://ar.qualcomm.at/node/2001624>
Is the 1.5 release the one you were talking about?
I'm all for having it easier! :)

Re: "Water (Pro Only)" Unity package reflection bug

February 21, 2012 - 5:57am #12

As a follow-up:
I just posted about this bug on Unity Answers.
The only new element I have to work on for now is that I noticed there were leaks even without displaying any 3D augmentation when profiling with Instruments.

Re: "Water (Pro Only)" Unity package reflection bug

February 8, 2012 - 8:57pm #11

Okay I have the project and I see the bug you're talking about. Very strange!

Here's a Unity thread that seems to be about the same topic: http://forum.unity3d.com/threads/79482-OpenGLES-error-0x0506-in-PresentSurface-677

I tried it with the Unity 3.5 beta but unfortunately that didn't help. Also tried it in Portrait as someone in that thread suggested, without luck. I'm a bit stumped, I'll look into more when I get a chance but I don't think it's on our end...

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 7, 2012 - 9:07pm #10

mediafire seems to be down right now, I'll try again tomorrow.

I'll admit I haven't tried the water effects on iOS yet, only on Android.

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 7, 2012 - 8:28am #9

http://www.mediafire.com/?l9lxoqpx3ixo5sk
I can't reproduce the bug on iPod, only on iPad.

EDIT: another link: http://www.2shared.com/file/Mp0JezWq/VBA.html

Re: "Water (Pro Only)" Unity package reflection bug

February 6, 2012 - 8:06pm #8

This part is rather challenging, especially since you don't know the FOV of the camera (QCAR sets this under the hood). One approach is to do some raycasting to find the bounds of the far clip plane and fit the mesh to that.

Here's a snippet that might help:

// Scale game object for full screen video image:
gameObject.transform.localScale = new Vector3(1, 1, 1 * (float)mTextureInfo.imageSize.y / (float)mTextureInfo.imageSize.x);

// Position the mesh at the far end of the perspective frustum

// Choose a point almost at the far clip plane
float dist = m_Camera.farClipPlane * 0.99f;

// Define a plane at the chosen distance
Plane farPlane = new Plane(m_Camera.transform.forward, m_Camera.transform.position + m_Camera.transform.forward * dist);

// Cast a ray along the lower left and upper right edges of the frustum
Ray ray1 = m_Camera.ScreenPointToRay(new Vector3(0, 0, 0));
Ray ray2 = m_Camera.ScreenPointToRay(new Vector3(m_Camera.pixelWidth, m_Camera.pixelHeight, 0));

// Find the points at which the rays intersect the plane
float rayDist = 0.0f;
farPlane.Raycast(ray1, out rayDist);
Vector3 p1 = m_Camera.transform.InverseTransformPoint(ray1.GetPoint(rayDist));
farPlane.Raycast(ray2, out rayDist);
Vector3 p2 = m_Camera.transform.InverseTransformPoint(ray2.GetPoint(rayDist));

// Position the video mesh at the chosen distance
gameObject.transform.localPosition = new Vector3(0, 0, dist);

// Scale the video mesh to stretch between the intersection points
gameObject.transform.localScale *= (p2.x - p1.x) / 2.0f;

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 6, 2012 - 8:30am #7

Hello Kim,

Thanks for the suggestion, it works a treat.

I still have issues with setting up the background plane that is textured by the video. I had previously set its scale with hard coded values so that it would approximately fit the screen.
But I'm trying to do it properly now. Using basic trigonometry in a right triangle, I'm able to scale the plane so it fits the camera field of view. But that only works in the editor.
I tried disabling the video-related code in VideoTextureBehaviour.cs and the camera projection matrix update in QCARBehaviour.cs. That allows me to see a properly scaled background plane, but I can't get any further. I realized that the mesh creation in VideoTextureBehaviour.cs could not be deleted to properly display the video, after reading this thread: http://ar.qualcomm.at/node/2001376.
I also read the Camera projectionMatrix Unity doc page, which only confirms my lack of knowledge on the subject... :) :
"
Use a custom projection only if you really need a non-standard projection. This property is used by Unity's water rendering to setup an oblique projection matrix. Using custom projections requires good knowledge of transformation and projection matrices.
"

To sum up the issue:
How do I properly scale the background plane so that its video texture exactly fills the screen on both iPhone and iPad?

Thank you!

EDIT: forgot to mention that the background plane is a child to the ARCamera

Re: "Water (Pro Only)" Unity package reflection bug

February 2, 2012 - 8:24pm #6

Ah, you're right! This is an interesting problem... Our DepthMask is typically used to punch through the entire Unity rendered scene, to allow the natively drawn camera image to show through.

Here's an idea: try drawing your video background using a shader in the Background render queue. I just made a copy of the Unlit/Texture shader and changed the tags:

Tags { "Queue" = "Background" }

There may be a shader that is already set up correctly for this, I'm just not sure which one :) Regardless, this custom shader seems to resolve the problem for me. With your video background plane using this, you should be able to use our DepthMask material like before.

For more on render queues: http://unity3d.com/support/documentation/Components/SL-SubshaderTags.html

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 2, 2012 - 8:46am #5

Ok, I had it wrong, I thought that solving the first issue was enough for what I needed, and that part 2 allowed to see the video image through the water.
After much guess work, I now have only the ARCamera active, with the video image textured on a plane in the background, near the camera far plane and scaled to match the camera field of view.

New problem is combining this with the issue described in this thread:
http://ar.qualcomm.at/node/2001379.
I'm trying to render a swimming pool and need to obscure its outer walls. But the DepthMask shader now also prevents parts of the video image from being rendered.

Sorry if I'm not making myself clear enough or not using the right vocabulary...

Thanks again!

Guillaume

Re: "Water (Pro Only)" Unity package reflection bug

February 1, 2012 - 8:25pm #4

With the default VBA sample the camera clear color is actually quite important. There are two cameras in the scene, the BackgroundCamera which renders the video and the ARCamera which renders the augmentation. The BackgroundCamera is rendered first. The ARCamera should not clear the color buffer, or it will overwrite the video drawn by the first camera.

That's why you need to draw both the video and the augmentation using the same camera for the water effects to work properly.

- Kim

Re: "Water (Pro Only)" Unity package reflection bug

February 1, 2012 - 2:21am #3

Hello,

Thanks for the quick reply.

I've been trying to get the water reflection to work in the VBA sample, but atm I'm stuck. Either I see the background video, but the reflection is buggy as before. Or I set the ARCamera clear flag to SolidColor instead of Depth (on line 404 of QCARBehaviour.cs), which fixes the reflection bug, but then I can't see the background video. The screen is black, apart from the augmentation when I point the camera at the Tarmac marker.
I've tried playing around with the AR camera background color RGBA values, to no avail.
It seems like the camera's background color alpha value is not used by Unity.

Re: "Water (Pro Only)" Unity package reflection bug

January 30, 2012 - 7:02pm #2

This is something we worked on just recently :) There are a couple of issues:

1) The Unity water effects seem to depend on the background clear color. Our QCARBehaviour.cs script overrides the camera clear settings of the ARCamera to allow the video background to be rendered natively. You can render the video background yourself using a method similar to the VideoBackgroundAccess sample, and then update the QCARBehaviour script to clear the background using a solid color. Look for "camera.clearFlags" in that script to find where it is set.

2) If you want the water effect to blend with the video image as well, you'll need to render the video and the augmentation using the same camera object. Right now the VideoBackgroundAccess sample uses two cameras: a perspective camera for the augmentation and an orthographic camera for the video image. You'll need to render the video image using the perspective camera (ARCamera) for everything to work correctly.

Our next release will have a few improvements to make all this easier, but you can try it yourself in the meanwhile.

- Kim

Log in or register to post comments