I hope they update or fix all bug on region capture for vuforia 8.1 

Hi guys.

1. Thank you all for your help.

2. For the new Vuforia 8. * you need to use: a) VideoBackground material instead of texture, as in the response of Rilissimo. b) use Renderer.sharedMaterial instead of Renderer.material to fix the problem with backgroundImage hanging on Android.

3. I refactored the source Region Capture plugin, but basically, everything is fine.

Quote:

This is done with Shader.EnableKeyword to set the keyword globally for the whole shader and with Material.EnableKeyword to set the keyword for the material only. The hard part is to find out what keywords to use. I couldn't find any function in the Vuforia API to obtain that information, but you could use Material.shaderKeywords on the VideoBackground Material to get the right shader keywords.

Do not do this if you plan on starting/stopping vuforia

https://developer.vuforia.com/forum/unity/bug-accessing-background-plane-texture-bug

 

And workaround

https://developer.vuforia.com/forum/unity/suggested-workaround-when-accessing-unity-materials-interferes-background-texture-updates



There is a way to get the PIXEL_FORMAT but I can't remember how I did it (and have since scrapped that solution because of the bug above so I can't find it now)

It should have something to do with how you setup the camera, but I could be wrong.

Edit: for search purposes PIXEL_FORMAT is an enum found in the Vuforia.Image class, the first thing that comes up in a web search is here

https://library.vuforia.com/content/vuforia-library/en/articles/Solution/How-To-Migrate-a-Unity-Project.html



Search for pixel_format in there, it might give you a clue

Rilissimo wrote:

you can actually use the same shader that is used for the video background from vuforia, you just need to set the keywords for the shader from code based on an enumeration from vuforia that tells you which texture format is being used.

Thanks Rilissimo, that info finally helped me to understand the issue and find a solution!

For all others, I will try to explain it a little more in depth:

What changed in the new Vuforia versions is that the camera feed can have diferrent Pixel Formats. I think before it was RGB only and now it could also be YUV in different variations on some devices. I experienced YUV being used on most Android devices and RGB being used on iOS and on my Mac Editor (maybe due to Metal?). So I had the Red Texture Problem only on Android devices.

YUV stores luminance information in the Y channel and chrominance information in the U and V channels. As Rilissimo stated we can have a look at the Vuforia VideoBackground Shader to see how this is implemented.

The first big change is that the Shader is using two Textures now: _MainTex and _UVTex.

[NoScaleOffset] _MainTex("Texture", 2D) = "white" {}
[NoScaleOffset] _UVTex("UV Texture", 2D) = "white" {}

_MainTex will contain the RGB information when the RGB Pixel Format is used. When the YUV Pixel Format is used, the _MainTex will contain only the Y channel and the second Texure _UVTex will contain the chrominance UV channels.

half2 uv = tex2D(_UVTex, i.uv2).rg;
float y = tex2D(_MainTex, i.uv).r;

As you can see the data is always stored in the first channels: Y in the red channel of _MainTex and UV in red and green in the _UVTex. This is also the explanation for the Red Texture Problem. Since Region_Capture is not updated yet it is only using one RGB texture in the RC_Simple_Color Shader. By default Region_Capture copies the color information from the BackgroundPlane, but when YUV is in use, the VideoBackground Shader will only store the luminance information in the red channel of the _MainTex. So only the luminance in the red channel is being copied to Region_Capture causing the Red Texture Problem.

The easiest solution is to skip the RC_Simple_Color Shader, which doesn't support the second UV Texture, yet. I don't see any reason why there is a need for a separate shader when we already have the VideoBackground Shader doing the color conversion work for us. We can simply take the VideoBackground Material and assign that to Region_Capture as Rilissimo already noted. You will find that solution in the TL;DR section below.

If you would want to keep the old RC_Simple_Color Shader and adjust it to support YUV, you would need to add the second _UVTex Texture as well as the different color conversion versions for YUV. Also, when using the adjusted shader, you would need to set the right ShaderKeywords to tell the shader / material which color conversion to use. Rilissimo already noted that as well. This is done with Shader.EnableKeyword to set the keyword globally for the whole shader and with Material.EnableKeyword to set the keyword for the material only. The hard part is to find out what keywords to use. I couldn't find any function in the Vuforia API to obtain that information, but you could use Material.shaderKeywords on the VideoBackground Material to get the right shader keywords.

TL;DR

In RegionCapture.cs replace line 83:

GetComponent<MeshRenderer>().material.mainTexture = VideoBackgroundTexure;

with:

GetComponent<MeshRenderer>().material = BackgroundPlane.GetComponent<MeshRenderer>().material;

I hope this helps some of you guys. Let me know if there are any issues with this solution. I won't promise to fix anything, but I'd like to know of course. ;)

And thanks again to Rilissimo!

Cheers,

Chris

you can actually use the same shader that is used for the video background from vuforia, you just need to set the keywords for the shader from code based on an enumeration from vuforia that tells you which texture format is being used.



I don't remember how to do it because I rolled back to a previous vuforia version, and now that we upgraded again I opted for another less optimized solution, basically rerendering the background plane on a different camera since accessing the real background plane properties will bug vuforia.

Well then those developers need to adapt or perish, if you want someone else to do your work for free look somewhere else.

I have already said enough on how to fix the issue, the rest is up to you all.

Rilissimo wrote:

Guys please read the answers below, I have already answered this, the textures are red  because RegionCapture assumes the texture is in RGB format, and this was true for the video background before vuforia 8, now you have to convert from YUV to RGB in the region capture shaders, use the VideoBackground shader from vuforia as an example on how to convert the values.

what name of shaders in the region capture ? 

because there is 4 shaders in Region_Capture folder

RC_ImageTarget_Dept_Sort, RC_MipMap_Blurred, RC_Simple_Color and RC_Transparent_Outer_Bounds

 

And yet, your answer is useless as most developers aren't familiar with shaders and how stuff converts there.

Guys please read the answers below, I have already answered this, the textures are red  because RegionCapture assumes the texture is in RGB format, and this was true for the video background before vuforia 8, now you have to convert from YUV to RGB in the region capture shaders, use the VideoBackground shader from vuforia as an example on how to convert the values.

I'm still having the same issue. Red Textures on Android with Vuforia 8.1.11 and Unity 2018.4.0.

It might have something to do with the OpenGL rendering on Android and textures being treated differently, but I couldn't figure out a solution yet.

I tried to access the background texture directly via VuforiaRenderer.VideoBackgroundTexture and give it to RegionCapture, but could net get it to work.

Does anyone have a clue or example how to obtain that Texture? I assume this will also be a red texture and therefore it would definitely be a Vuforia bug.

HI

i have some problem, i get textures from RenderTextureCamera are rendered "reddish", anyone know how to fixed this issue?

in this case i using vuforia 8.1.10 and unity 2018.3.14 for android platform.

here the step :

- After add ARcamera and import vuforia,

- setting xr for vuforia,

- add Capture_Region layer,

- swith to android,

- and import capture region package, and then try sample(get texture) and build to apk file

here the result (attach)...i get textures  rendered "reddish" from RenderTextureCamera. 

note: why video videobackground is small , because  videobackground in ARcamera is disable

I'm using 8.1.10 and the issue was in that version.

What you are describing happens only in 8.0.10, then fixed in 8.1.7



But something else broke in 8.1.7 (https://developer.vuforia.com/forum/unity/bug-accessing-background-plane-texture-bug)



My advice is to just stick with 7.5.26 until we get a more stable release

So I'm dealing with the same issue and it seems from examining the BackgroundPlane video texture and the material on the Region_Capture that the video texture is inset with large areas of black on the top and on the right. As a temporary measure I set up a secondary Region Capture that takes the un-uv mapped texture and renders that to a texture, and then crop the resulting texture excluding the, what seems to me, consistent black areas, and then rather than having the video texture applied to the main region capture I'm applying the cropped texture to that mesh. I'm sure there's a better way to do this to figure out why Android is adding black areas to the video texture when applied to a mesh but I haven't been able to figure it out. Unity 2019/Vuforia8.10/Region Capture latest.

Thanks maximrouf for a great plugin.  Are there any planned fixes for the Android shift in the latest version?

• .2.0a13
• Vuforia 8.0.10
• RegionCapture - latest

Doesn't anyone try vuforia version 8.0.10? I just try it and region capture still not working in android

Doesn't anyone find out how to fix region capture yet? i really need this to finish my project, please help!!!

Ok enough people asked me in messages to post the solution here so now I feel bad for not sharing, damn you all.

Short story, you need to convert the background plane texture format to the appropriate one based on the platform it is running on.

Easiest way is to use the background plane shader to render the region capture mesh (or you can use the conversion maths in there to edit your currently used shaders)

after this you'll still have to set the correct shader keywords, https://docs.unity3d.com/ScriptReference/Shader.EnableKeyword.html



in the latest vuforia version notes it tells you how to check what texture format vuforia is currently using.

After this you should all be set, hoping i didn't miss anything since I don't have the solution at hand right now, I had to revert the whole project to the previous version of unity and vuforia because they totally messed up the background plane texture acquisition. it works fine if you access it once in the whole lifetime of your app, but if you start and stop vuforia after accessing it it won't work anymore.

I found a fix

And I was writing about my journey to fix this.

But someone deleted my post, so good luck all.

michalSBIT wrote:

Hello,

 

after upgrade to Unity 2018.3 and Vuforia 8.1.7 RC textures from RenderTextureCamera are rendered "reddish". Screenshot attached. It's standard white paper with black ink print on it.

It occurs only on mobile devices (both Android and iOS), no problem when trying out in Untiy Editor. Had no troubles using older vuforia and Unity 2017.

 

Anyone facing similar problems?

Hello,

I am having the same issues with a red texture on Android (LG G4), but not on iOS (iPhone 6) and not in the Editor.

What I found out so far is that the following line is where the 'magic' happens:

Line 71 RegionCapture.cs:

VideoBackgroundTexure = BackgroundPlane.GetComponent<MeshRenderer>().material.mainTexture;

The Texture of the BackgroundPlane (containing the video feed) is shown correctly as background video, but when assigned / referenced to another Material it turns red. Does anyone have a clue why this might happen? An unsupported color format of the texture maybe? And if so, can it be converted?

The only possible solution I can imagine right now is not to use the BackgroundPlane texture in the RegionCapture script and instead reference our own WebCamTexture, but I have not tested that yet.

Also, even when the Video Background in the VuforiaConfiguration is disabled, a BackgroundPlane with the camera feed is attached to the ARCamera (just not fullscreen in the Editor) and the Video Background is still visible. Seems like a bug to me.

I apologize. I only asked because I saw some relevant references on older posts.

Anyway, on a Vuforia question, does the "freeze" feature works well on everyone? Changing the bool value on the "Freeze Enable" variable doesn't seem to make any difference on Unity 2018 with the latest Vuforia version.

Hello @vufer1980,

A friendly reminder that this is a support community for the Vuforia Engine SDK, Unity Extension, associated tools and sample applications.

I suggest you refer your experiences and questions for EasyAR to their forum: https://forum.easyar.com/portal.php?mod=list&catid=1

Thanks,

Vuforia Engine Support

Did anyone manage to get this to work with EasyAR in android?

I've tried using the "modified" unitypackage especially for EasyAR and put the RenderCamera with the RealityPlane in the appropriate slots but as soon as I compile and run to my android device I get a black background (not the camera feed) and the AR object appears without the camera capture material, just the material I have applied inside the Unity Editor.

I used Unity 2018.2.15.f1 and RegionCapture2.0.6_EasyAR.

For what it's worth it works kinda ok in the Editor since I understood I had to modify the marked parts in the RegionCapture.cs

I also have similar problem in unity 2018.3 .. :(

maybe anyone have solution?

Hello,

after upgrade to Unity 2018.3 and Vuforia 8.1.7 RC textures from RenderTextureCamera are rendered "reddish". Screenshot attached. It's standard white paper with black ink print on it.

It occurs only on mobile devices (both Android and iOS), no problem when trying out in Untiy Editor. Had no troubles using older vuforia and Unity 2017.

Anyone facing similar problems?

Hello @pmullins3,

Can you confirm that you're issue is similar to this one?: https://developer.vuforia.com/forum/issues-and-bugs/vuforia-upgrade-installer-breaks-support-users

Thanks,

Vuforia Engine Support

Hi,

I'm having trouble updating unity to version 8.0.10 - the installer just seems to remove all reference of Vuforia from the editor.

Anyone had the same problem? I'm using unity 2018.2.21f1 via unity hub.

Any help would be much appreciated!

Thanks.

Same issue here..! Does someone fixed this??

Unity 2018.3.6f1 / Vuforia 8.0.10 / Region Capture 2.0.9

How did you manage to implement it in ARToolKit? Any instructions or source code perhaps? Thanks!

Same issue. RegionCapture is not wrapping on 3D objects properly

Unity 2018.3.6f1, Vuforia 8.0.10, Region Capture 2.0.9

But wraps properly on Unity 2018.2.20f1, Vuforia 6.2-10, Region Capture 1.8.0

This android problem is really frustrating :( 



Hope someone psots a fix for it here soon. Still not working.

Same issue here, working in editor but not in android device, any workaround?

Same issue with region capture not working with Android.

Unity 2018.2.5f1, Vuforia 8.0.10, Region Capture 2.0.9

Works on iOS and in Unity Editor, just not on Android devices.

Region capture no longer wrapping on 3D object correctly. (worked fine in earlier version of Unity and Vuforia)

Please fix this bug : " Vuforia "8.0.10". The AR coloring (REGION CAPTURE, GET-TEXTURE) works in editor but not in android.

I have same issue :(

Hi Maximrouf,



I adopted "Region_Capture" before and it works great.

https://github.com/maximrouf/RegionCapture.

But after i update my unity "2018.3.1" and vuforia "8.0.10". The AR coloring now works in editor but not in android. 

Could you please suggest how to solve it. Many many thanks. I am in urgent.

Hi maximrouf,

First of all, I would like to thank you for this amazing plugin !

BTW, I'm currently working on porting an app on Universal Windows Platform. Everything works find in Unity Editor, but not when I build as an appx, RegionCapture seems to return an empty texture.

Here is my configuration :

Unity version : 2018.2.6f1 (64bit)

Vuforia version : 7.2.23

RegionCapture version : 

Device name : MSI GS65

Device OS version : Windows 10 Pro

Orientation of the device : Landscape

Thanks !

Hello all,

I can run Region Capture, but the problem is Live Texuturing is not working properly. If I try to color the 3d model at bottom it colors slightly up from there. And also if I try to color at the top it colors bottom of the 3d model.

Please help as soon as possible. As I'm trying since so many days still not getting any solution.

Thanks in advance.

Hello maximrouf,

Your plugin is great as it looks.

But unfortunately live coloring is not working for me.

I'm using Unity 2018.2.1f1, Vuforia 7.5.20 and Region Capture 2.0.9.

I'm running it on iPhone 7 and iOS 12.1.

Want a help on this...

Thanks in advance if anyone could help.

Hi! Does this work with multiple image targets including ones that don't need region capture? It seems camera world center mode is defined by the first image... 

Hi @maximrouf,

I tried to edit the script for auto stop capturing after first tracking found,

I want to disable "Stop Capturing" button and change the trigger to script but it doesn't success.

Any example for that?

Thanks 

I fixed the issue by updating Unity and Vuforia.

Unity 2018.211f1 with Region Capture v2.0.9 and Vuforia V7.2.23

Best regards

Hi @maximrouf,

I'm using Unity 2018.2.2f with Region Capture v2.0.9 and Vuforia V7.1.35

I encountered an issue on iOS where the render camera seems to stop capturing after flipping the camera via Vuforia API:

        CameraDevice.Instance.Stop();

        CameraDevice.Instance.Deinit();

        bool result = CameraDevice.Instance.Init(newDirection);

        CameraDevice.Instance.Start();

        Debug.Log((result) ? "Successfully changed" : "Failed to change cam");

        VuforiaUnity.OnPause();

        VuforiaUnity.OnResume();

        return result;

Here are the steps:

1. Start camera (Default = back camera)

2. Scan image target, model appears with the region mapped as texture correctly.

3. Flip the camera (now = front camera)

4. Flip the camera again (now = back camera)

5. Scan image target, notice that the model appears to have a distorted texture.

In step 6, the texture that seems to be mapped is the last camera frame captured before the first camera flip happens. 

Any help will be appreciated.

Thanks.

Hi @maximrouf, @Pavan1607,

Do you have any work around on solving this issue? I'm facing the same problem on IOS Devices.

On iOS Region_Capture prefab is not aligned to the image target. It moves when the camera is moved .

Faced the issue on iPad pro and iPhone 7

Vuforia 7.0.36, Region Capture 2.0.9, Unity 2017.3.0f3

Any help will be appreciated.

SOLVED: 09/10/2017

Solved using Unity 2018.2.11f1 with Region Capture v2.0.9 and Vuforia V7.2.23

tomtomtongtong wrote:

thienbaomoon wrote:

Dear maximrouf,

I am using Vuforia 7.1.35, Region Capture 2.0.9, Unity 2018.2.

I want to create a coloring page, I've tried to run the sample scene and the texture renders the camera instead of the image target.

Could you give me some suggestion for this?, Please

here is the video showing the problem.

https://youtu.be/nPkWOTWXWNw

 

Yes. I also meet this issue in Unity 2018.2

I tried below tutorial with Unity 2018.2 and Vuforia 7.1.35 and it seemed to be working. If you still face any problem then contact me on sushanta1991(@)rediffmail(dot)com

https://sushanta1991.blogspot.com/2018/07/how-to-make-ar-coloring-application-in.html

thienbaomoon wrote:

Dear maximrouf,

I am using Vuforia 7.1.35, Region Capture 2.0.9, Unity 2018.2.

I want to create a coloring page, I've tried to run the sample scene and the texture renders the camera instead of the image target.

Could you give me some suggestion for this?, Please

here is the video showing the problem.

https://youtu.be/nPkWOTWXWNw

Yes. I also meet this issue in Unity 2018.2

If anyone wants to use region capture with vuforia, below link explains it all how you can create an ar sketch application.

link

Dear maximrouf,

I am using Vuforia 7.1.35, Region Capture 2.0.9, Unity 2018.2.

I want to create a coloring page, I've tried to run the sample scene and the texture renders the camera instead of the image target.

Could you give me some suggestion for this?, Please

here is the video showing the problem.

https://youtu.be/nPkWOTWXWNw

Hello,

I am using Unity 2017.3.0 and Region capture Ver. 2.0.9. The region capture is not working on the Vuforia version. I am trying to create a AR coloring page. Not able to find a few settings in Victoria as shown in the tutorial video. Does anyone have solution for this?

Thanks

Hello, 

I am using Unity 2017.3.0 and Region capture Ver. 2.0.9. 

I am facing a weird problem, when I am using my model, it is not getting live texture but when I use default models like cube and Sphere, I can see the live texture on them. 

Can anybody have solution for this. 

I am attaching screenshot for the reference. 

Thanks,

Vanshika 

Hi everyone,

I want to be able to save multiple textures and reapply them to my model when I reopen the app.

I already know how I can actually read in all the pixels of a texture and how it is encoded to PNG. And saved afterwards.

All of this can be achieved using the MakeScreen() function of RenderTextureCamera.cs class.

We know that the model is being fed this type of information when its texture gets changed live.

How can I feed the model this saved pixels information?

As everyone can guess, it does not have to be live when remodeling from a saved texture.

It can just be one function where it takes the saved pixels information and applies the info to the model.

Thanks!!!