Hey all ,

Can anyone will explain the working of region capture ? I tried to read the code but understood it partially i got basic understanding of some functions in "Region Capture Script"  but i didn't got any intuition  of what is the control flow of program .. can anybody help me out ? I just want to get the basic workflow of how the script is retrieving the image target at runtime. 

Hey, could anyone help me. I'm using unity 2019.3.14f, vuforia 9.1.7 and region capture 2.2.5. I already imported all those on unity. But I couldn't find AR camera as shown in this video. How do i find that? This is my final year project. Hoping that anyone could really help me. Thank you! 

https://youtu.be/54W5hF-ZnO8

I can't get it working with different screen aspect ratios. 4:3 is the only that works for me. For all other ratios, the captured image is strongly distorted :(

Is this a bug?

EDIT:

Okay so I had a closer look on that and I managed to fix the distorted image on Android.

The problem is that the aspect ratio of your camera video feed is another than the ratio of your screen. For me it's 2240x1080 for the screen and 4:3 for the video feed. If you give the region capture the ARCamera to use WorldToViewport, you get a distored image on your region capture plane. The ARCamera zooms in on the video feed in order to fill the whole screen of the phone.

The solution is to create another camera, on runtime set its position and rotation to the same as of the ARCamera, give it the correct field of view and aspect ratio so that it fits your video feed, and then wire this camera to the region capture. Side note: I don't really know how you can calculate the field of view it, but I found mine out using the ARCamera temporarily setting the aspect ratio of the editor to the one from the video feed and copying the fov of the ARCamera which is set automatically by Vuforia.

Hope it helps,

Regards

gokhany wrote:

CAN SOMEONE HELP ME?

for me it has to do with the aspect ratio of the screen of my mobile. You can recreate it in Edit Mode when you use the same screen resolution/aspect ratio.

Hi,

Can anyone confirm the latest version 2.5 works for Android 64 bit? Our app needs update and we experienced red camera or rotation issues with older version around 1 year ago.

Best regards

maximrouf wrote:

Hello everyone. New version 2.2.2 (Final) on GitHub

Regards, Maxim Rouf

Unity2019,2,12 and 2,2,5 plugin. I add CameraOutputTexture from RenderTextureCamera.cs to RAW Image in UI. On Unity Editor region cupture works very well and my coloring app are work. Video -> https://youtu.be/JXcP41MrRok

But when i build on Android i have big problems. My Raw Image too bad and coloring app doesnt work(( Video -> https://youtu.be/tt_hFBEAeaA 

I Think its problem because of  FOW AR Camera.

Hello there, i tried to develop an AR coloring book. everything works fine, but after some check, try and error, testing, there is some issue i get which is during changing app and try to use back the coloring AR, the camera start to revert back to flip x or rotate 90. So if u can do a favor and help me figure out the problem, it would be much helpful.

Thank you for read this question. All the best

Regard,

Airil

Hello everyone. New version 2.2.2 (Final) on GitHub

Regards, Maxim Rouf

- Unity 2019.2.8f1

- Vuforia 8.3.9

- Region Capture 2.2.1

- Android

Render Camera not capturing region of my marker in android. In editor it's perfectly captured but when i tried on device looks like this https://ibb.co/vcxxyMJ

The raw image on left bottom screen showing image from RenderTextureCamera.GetRenderTexture()

I placed the RegionCapture prefab exactly same as the marker position and scale.

- Unity 2019.2.5f1

- Vuforia 8-5-8

- Region Capture 2.2.1

- Device: Xiaomi mi A2 Lite

- Android

- Both (Portrait and landscape)

I have an issue about material. My duck looks red.

Here is a screenshot from my app: https://ibb.co/tmbtMjn

Hello Everyone,

I'm using :

- Unity 2018.4.11f1 (LTS)

- Vuforia 8.3.8

- Region Capture 2.0.9

I use an Image Target to make my 3D models appears on it. When i tracking the marker all working fine, but when i leave the app and return on it, the camera freeze and all 3D models "flying".

Have you got some idea, how to avoid this ???



Thank you so much guys !

gokhany wrote:

nadzim96 wrote:

klelvin wrote:

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

 

Hello all, 

Same with klelvin, im also facing the same problem. Could someone please let me know whether there is any update on how to fix this problem. Here is the image on this issue.

 

Thank you

 

I have same issue. can someone help now?

CAN SOMEONE HELP ME?

unity 2019.2.8 vuforia 8.3.8 build on android (samsung a30)

Hi maximrouf,

I`m Using Unity 2019.1.14 and Vuforia 8.3.8. I downloaded the new version of Region Capture on GitHub and open an example scene.

When I build the App to an Android device, the texture remains red as before.

Do you have any idea what the problem is?

Hello everyone. New version 2.2.0 (beta) on GitHub

Regards, Maxim Rouf

nadzim96 wrote:

klelvin wrote:

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

 

Hello all, 

Same with klelvin, im also facing the same problem. Could someone please let me know whether there is any update on how to fix this problem. Here is the image on this issue.

 

Thank you

I have same issue. can someone help now?

Hi there, we are running into the same issue and I was wondering if you ever found a fix or work around? If so, I would really appreciate some support. If it is a bigger undertaking, I'd be happy to pay you for your time. 

Thanks!

We had the same issue and had to use an older version of vuforia, however, due to new requirements from Google for 64 bit builds, we had to update Vuforia and the coloring plugin stopped working on the new Vuforia. We haven't figured it out as to why it only effects Android in the new version of Vuforia and we've been picking away at it to figure out the difference. Unfortunately, the plugin is not officially support by Vuforia and the guy who made it did so in his free time, so we may never see an update from him again on it. 

klelvin wrote:

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

Hello all, 

Same with klelvin, im also facing the same problem. Could someone please let me know whether there is any update on how to fix this problem. Here is the image on this issue.

Thank you

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

kpprt wrote:

There's another bug on Android caused by the VideoBackgroundTexure reference in Region_Capture.

To reproduce: on Android do the following:

1. Start your App, initialize Vuforia
2. Scan one of your image targets with Region_Capture
3. Pause the App, e.g. by clicking the app overview button
4. Resume the App by clicking on it in the app overview

Result: the background video texture freezes

Expected: the background video texture doesn't freeze and continues to show the camera image

Solution/Workaround:

Since we don't need the VideoBackgroundTexure reference any more when using the SharedMaterial directly (like described in the previous posts), make sure to comment out all occurences of VideoBackgroundTexure in RegionCapture.cs and RegionCaptureEditor.cs.

Cheers,

Chris

Hello,

Your approach didn`t work for me. I use Unity 2018.4.3 Vuforia 8.1.10

I delete all reference VideoBackgroundTexure. Do you know any solutions?

There's another bug on Android caused by the VideoBackgroundTexure reference in Region_Capture.

To reproduce: on Android do the following:

1. Start your App, initialize Vuforia
2. Scan one of your image targets with Region_Capture
3. Pause the App, e.g. by clicking the app overview button
4. Resume the App by clicking on it in the app overview

Result: the background video texture freezes

Expected: the background video texture doesn't freeze and continues to show the camera image

Solution/Workaround:

Since we don't need the VideoBackgroundTexure reference any more when using the SharedMaterial directly (like described in the previous posts), make sure to comment out all occurences of VideoBackgroundTexure in RegionCapture.cs and RegionCaptureEditor.cs.

Cheers,

Chris

I hope they update or fix all bug on region capture for vuforia 8.1 

Hi guys.

1. Thank you all for your help.

2. For the new Vuforia 8. * you need to use: a) VideoBackground material instead of texture, as in the response of Rilissimo. b) use Renderer.sharedMaterial instead of Renderer.material to fix the problem with backgroundImage hanging on Android.

3. I refactored the source Region Capture plugin, but basically, everything is fine.

Quote:

This is done with Shader.EnableKeyword to set the keyword globally for the whole shader and with Material.EnableKeyword to set the keyword for the material only. The hard part is to find out what keywords to use. I couldn't find any function in the Vuforia API to obtain that information, but you could use Material.shaderKeywords on the VideoBackground Material to get the right shader keywords.

Do not do this if you plan on starting/stopping vuforia

https://developer.vuforia.com/forum/unity/bug-accessing-background-plane-texture-bug

 

And workaround

https://developer.vuforia.com/forum/unity/suggested-workaround-when-accessing-unity-materials-interferes-background-texture-updates



There is a way to get the PIXEL_FORMAT but I can't remember how I did it (and have since scrapped that solution because of the bug above so I can't find it now)

It should have something to do with how you setup the camera, but I could be wrong.

Edit: for search purposes PIXEL_FORMAT is an enum found in the Vuforia.Image class, the first thing that comes up in a web search is here

https://library.vuforia.com/content/vuforia-library/en/articles/Solution/How-To-Migrate-a-Unity-Project.html



Search for pixel_format in there, it might give you a clue

Rilissimo wrote:

you can actually use the same shader that is used for the video background from vuforia, you just need to set the keywords for the shader from code based on an enumeration from vuforia that tells you which texture format is being used.

Thanks Rilissimo, that info finally helped me to understand the issue and find a solution!

For all others, I will try to explain it a little more in depth:

What changed in the new Vuforia versions is that the camera feed can have diferrent Pixel Formats. I think before it was RGB only and now it could also be YUV in different variations on some devices. I experienced YUV being used on most Android devices and RGB being used on iOS and on my Mac Editor (maybe due to Metal?). So I had the Red Texture Problem only on Android devices.

YUV stores luminance information in the Y channel and chrominance information in the U and V channels. As Rilissimo stated we can have a look at the Vuforia VideoBackground Shader to see how this is implemented.

The first big change is that the Shader is using two Textures now: _MainTex and _UVTex.

[NoScaleOffset] _MainTex("Texture", 2D) = "white" {}
[NoScaleOffset] _UVTex("UV Texture", 2D) = "white" {}

_MainTex will contain the RGB information when the RGB Pixel Format is used. When the YUV Pixel Format is used, the _MainTex will contain only the Y channel and the second Texure _UVTex will contain the chrominance UV channels.

half2 uv = tex2D(_UVTex, i.uv2).rg;
float y = tex2D(_MainTex, i.uv).r;

As you can see the data is always stored in the first channels: Y in the red channel of _MainTex and UV in red and green in the _UVTex. This is also the explanation for the Red Texture Problem. Since Region_Capture is not updated yet it is only using one RGB texture in the RC_Simple_Color Shader. By default Region_Capture copies the color information from the BackgroundPlane, but when YUV is in use, the VideoBackground Shader will only store the luminance information in the red channel of the _MainTex. So only the luminance in the red channel is being copied to Region_Capture causing the Red Texture Problem.

The easiest solution is to skip the RC_Simple_Color Shader, which doesn't support the second UV Texture, yet. I don't see any reason why there is a need for a separate shader when we already have the VideoBackground Shader doing the color conversion work for us. We can simply take the VideoBackground Material and assign that to Region_Capture as Rilissimo already noted. You will find that solution in the TL;DR section below.

If you would want to keep the old RC_Simple_Color Shader and adjust it to support YUV, you would need to add the second _UVTex Texture as well as the different color conversion versions for YUV. Also, when using the adjusted shader, you would need to set the right ShaderKeywords to tell the shader / material which color conversion to use. Rilissimo already noted that as well. This is done with Shader.EnableKeyword to set the keyword globally for the whole shader and with Material.EnableKeyword to set the keyword for the material only. The hard part is to find out what keywords to use. I couldn't find any function in the Vuforia API to obtain that information, but you could use Material.shaderKeywords on the VideoBackground Material to get the right shader keywords.

TL;DR

In RegionCapture.cs replace line 83:

GetComponent<MeshRenderer>().material.mainTexture = VideoBackgroundTexure;

with:

GetComponent<MeshRenderer>().material = BackgroundPlane.GetComponent<MeshRenderer>().material;

I hope this helps some of you guys. Let me know if there are any issues with this solution. I won't promise to fix anything, but I'd like to know of course. ;)

And thanks again to Rilissimo!

Cheers,

Chris

you can actually use the same shader that is used for the video background from vuforia, you just need to set the keywords for the shader from code based on an enumeration from vuforia that tells you which texture format is being used.



I don't remember how to do it because I rolled back to a previous vuforia version, and now that we upgraded again I opted for another less optimized solution, basically rerendering the background plane on a different camera since accessing the real background plane properties will bug vuforia.

Well then those developers need to adapt or perish, if you want someone else to do your work for free look somewhere else.

I have already said enough on how to fix the issue, the rest is up to you all.

Rilissimo wrote:

Guys please read the answers below, I have already answered this, the textures are red  because RegionCapture assumes the texture is in RGB format, and this was true for the video background before vuforia 8, now you have to convert from YUV to RGB in the region capture shaders, use the VideoBackground shader from vuforia as an example on how to convert the values.

what name of shaders in the region capture ? 

because there is 4 shaders in Region_Capture folder

RC_ImageTarget_Dept_Sort, RC_MipMap_Blurred, RC_Simple_Color and RC_Transparent_Outer_Bounds

 

And yet, your answer is useless as most developers aren't familiar with shaders and how stuff converts there.

Guys please read the answers below, I have already answered this, the textures are red  because RegionCapture assumes the texture is in RGB format, and this was true for the video background before vuforia 8, now you have to convert from YUV to RGB in the region capture shaders, use the VideoBackground shader from vuforia as an example on how to convert the values.

I'm still having the same issue. Red Textures on Android with Vuforia 8.1.11 and Unity 2018.4.0.

It might have something to do with the OpenGL rendering on Android and textures being treated differently, but I couldn't figure out a solution yet.

I tried to access the background texture directly via VuforiaRenderer.VideoBackgroundTexture and give it to RegionCapture, but could net get it to work.

Does anyone have a clue or example how to obtain that Texture? I assume this will also be a red texture and therefore it would definitely be a Vuforia bug.

HI

i have some problem, i get textures from RenderTextureCamera are rendered "reddish", anyone know how to fixed this issue?

in this case i using vuforia 8.1.10 and unity 2018.3.14 for android platform.

here the step :

- After add ARcamera and import vuforia,

- setting xr for vuforia,

- add Capture_Region layer,

- swith to android,

- and import capture region package, and then try sample(get texture) and build to apk file

here the result (attach)...i get textures  rendered "reddish" from RenderTextureCamera. 

note: why video videobackground is small , because  videobackground in ARcamera is disable

I'm using 8.1.10 and the issue was in that version.

What you are describing happens only in 8.0.10, then fixed in 8.1.7



But something else broke in 8.1.7 (https://developer.vuforia.com/forum/unity/bug-accessing-background-plane-texture-bug)



My advice is to just stick with 7.5.26 until we get a more stable release

So I'm dealing with the same issue and it seems from examining the BackgroundPlane video texture and the material on the Region_Capture that the video texture is inset with large areas of black on the top and on the right. As a temporary measure I set up a secondary Region Capture that takes the un-uv mapped texture and renders that to a texture, and then crop the resulting texture excluding the, what seems to me, consistent black areas, and then rather than having the video texture applied to the main region capture I'm applying the cropped texture to that mesh. I'm sure there's a better way to do this to figure out why Android is adding black areas to the video texture when applied to a mesh but I haven't been able to figure it out. Unity 2019/Vuforia8.10/Region Capture latest.

Thanks maximrouf for a great plugin.  Are there any planned fixes for the Android shift in the latest version?

• .2.0a13
• Vuforia 8.0.10
• RegionCapture - latest

Doesn't anyone try vuforia version 8.0.10? I just try it and region capture still not working in android

Doesn't anyone find out how to fix region capture yet? i really need this to finish my project, please help!!!

Ok enough people asked me in messages to post the solution here so now I feel bad for not sharing, damn you all.

Short story, you need to convert the background plane texture format to the appropriate one based on the platform it is running on.

Easiest way is to use the background plane shader to render the region capture mesh (or you can use the conversion maths in there to edit your currently used shaders)

after this you'll still have to set the correct shader keywords, https://docs.unity3d.com/ScriptReference/Shader.EnableKeyword.html



in the latest vuforia version notes it tells you how to check what texture format vuforia is currently using.

After this you should all be set, hoping i didn't miss anything since I don't have the solution at hand right now, I had to revert the whole project to the previous version of unity and vuforia because they totally messed up the background plane texture acquisition. it works fine if you access it once in the whole lifetime of your app, but if you start and stop vuforia after accessing it it won't work anymore.

I found a fix

And I was writing about my journey to fix this.

But someone deleted my post, so good luck all.

michalSBIT wrote:

Hello,

 

after upgrade to Unity 2018.3 and Vuforia 8.1.7 RC textures from RenderTextureCamera are rendered "reddish". Screenshot attached. It's standard white paper with black ink print on it.

It occurs only on mobile devices (both Android and iOS), no problem when trying out in Untiy Editor. Had no troubles using older vuforia and Unity 2017.

 

Anyone facing similar problems?

Hello,

I am having the same issues with a red texture on Android (LG G4), but not on iOS (iPhone 6) and not in the Editor.

What I found out so far is that the following line is where the 'magic' happens:

Line 71 RegionCapture.cs:

VideoBackgroundTexure = BackgroundPlane.GetComponent<MeshRenderer>().material.mainTexture;

The Texture of the BackgroundPlane (containing the video feed) is shown correctly as background video, but when assigned / referenced to another Material it turns red. Does anyone have a clue why this might happen? An unsupported color format of the texture maybe? And if so, can it be converted?

The only possible solution I can imagine right now is not to use the BackgroundPlane texture in the RegionCapture script and instead reference our own WebCamTexture, but I have not tested that yet.

Also, even when the Video Background in the VuforiaConfiguration is disabled, a BackgroundPlane with the camera feed is attached to the ARCamera (just not fullscreen in the Editor) and the Video Background is still visible. Seems like a bug to me.

I apologize. I only asked because I saw some relevant references on older posts.

Anyway, on a Vuforia question, does the "freeze" feature works well on everyone? Changing the bool value on the "Freeze Enable" variable doesn't seem to make any difference on Unity 2018 with the latest Vuforia version.

Hello @vufer1980,

A friendly reminder that this is a support community for the Vuforia Engine SDK, Unity Extension, associated tools and sample applications.

I suggest you refer your experiences and questions for EasyAR to their forum: https://forum.easyar.com/portal.php?mod=list&catid=1

Thanks,

Vuforia Engine Support

Did anyone manage to get this to work with EasyAR in android?

I've tried using the "modified" unitypackage especially for EasyAR and put the RenderCamera with the RealityPlane in the appropriate slots but as soon as I compile and run to my android device I get a black background (not the camera feed) and the AR object appears without the camera capture material, just the material I have applied inside the Unity Editor.

I used Unity 2018.2.15.f1 and RegionCapture2.0.6_EasyAR.

For what it's worth it works kinda ok in the Editor since I understood I had to modify the marked parts in the RegionCapture.cs

I also have similar problem in unity 2018.3 .. :(

maybe anyone have solution?

Hello,

after upgrade to Unity 2018.3 and Vuforia 8.1.7 RC textures from RenderTextureCamera are rendered "reddish". Screenshot attached. It's standard white paper with black ink print on it.

It occurs only on mobile devices (both Android and iOS), no problem when trying out in Untiy Editor. Had no troubles using older vuforia and Unity 2017.

Anyone facing similar problems?

Hello @pmullins3,

Can you confirm that you're issue is similar to this one?: https://developer.vuforia.com/forum/issues-and-bugs/vuforia-upgrade-installer-breaks-support-users

Thanks,

Vuforia Engine Support

Hi,

I'm having trouble updating unity to version 8.0.10 - the installer just seems to remove all reference of Vuforia from the editor.

Anyone had the same problem? I'm using unity 2018.2.21f1 via unity hub.

Any help would be much appreciated!

Thanks.

Same issue here..! Does someone fixed this??

Unity 2018.3.6f1 / Vuforia 8.0.10 / Region Capture 2.0.9

How did you manage to implement it in ARToolKit? Any instructions or source code perhaps? Thanks!