Log in or register to post comments

Region Capture

May 31, 2019 - 11:28pm #36

And yet, your answer is useless as most developers aren't familiar with shaders and how stuff converts there.

Region Capture

June 2, 2019 - 6:23pm #35

Rilissimo wrote:

Guys please read the answers below, I have already answered this, the textures are red  because RegionCapture assumes the texture is in RGB format, and this was true for the video background before vuforia 8, now you have to convert from YUV to RGB in the region capture shaders, use the VideoBackground shader from vuforia as an example on how to convert the values.

 

what name of shaders in the region capture ? 

because there is 4 shaders in Region_Capture folder

RC_ImageTarget_Dept_Sort, RC_MipMap_Blurred, RC_Simple_Color and RC_Transparent_Outer_Bounds

 

Region Capture

June 3, 2019 - 12:56am #34

Well then those developers need to adapt or perish, if you want someone else to do your work for free look somewhere else.

I have already said enough on how to fix the issue, the rest is up to you all.

Region Capture

June 3, 2019 - 1:01am #33

you can actually use the same shader that is used for the video background from vuforia, you just need to set the keywords for the shader from code based on an enumeration from vuforia that tells you which texture format is being used.



I don't remember how to do it because I rolled back to a previous vuforia version, and now that we upgraded again I opted for another less optimized solution, basically rerendering the background plane on a different camera since accessing the real background plane properties will bug vuforia.

 

 

Region Capture

June 6, 2019 - 1:57am #32

Rilissimo wrote:

you can actually use the same shader that is used for the video background from vuforia, you just need to set the keywords for the shader from code based on an enumeration from vuforia that tells you which texture format is being used.

Thanks Rilissimo, that info finally helped me to understand the issue and find a solution!

For all others, I will try to explain it a little more in depth:

What changed in the new Vuforia versions is that the camera feed can have diferrent Pixel Formats. I think before it was RGB only and now it could also be YUV in different variations on some devices. I experienced YUV being used on most Android devices and RGB being used on iOS and on my Mac Editor (maybe due to Metal?). So I had the Red Texture Problem only on Android devices.

YUV stores luminance information in the Y channel and chrominance information in the U and V channels. As Rilissimo stated we can have a look at the Vuforia VideoBackground Shader to see how this is implemented.

The first big change is that the Shader is using two Textures now: _MainTex and _UVTex.

[NoScaleOffset] _MainTex("Texture", 2D) = "white" {}
[NoScaleOffset] _UVTex("UV Texture", 2D) = "white" {}

_MainTex will contain the RGB information when the RGB Pixel Format is used. When the YUV Pixel Format is used, the _MainTex will contain only the Y channel and the second Texure _UVTex will contain the chrominance UV channels.

half2 uv = tex2D(_UVTex, i.uv2).rg;
float y = tex2D(_MainTex, i.uv).r;

As you can see the data is always stored in the first channels: Y in the red channel of _MainTex and UV in red and green in the _UVTex. This is also the explanation for the Red Texture Problem. Since Region_Capture is not updated yet it is only using one RGB texture in the RC_Simple_Color Shader. By default Region_Capture copies the color information from the BackgroundPlane, but when YUV is in use, the VideoBackground Shader will only store the luminance information in the red channel of the _MainTex. So only the luminance in the red channel is being copied to Region_Capture causing the Red Texture Problem.

The easiest solution is to skip the RC_Simple_Color Shader, which doesn't support the second UV Texture, yet. I don't see any reason why there is a need for a separate shader when we already have the VideoBackground Shader doing the color conversion work for us. We can simply take the VideoBackground Material and assign that to Region_Capture as Rilissimo already noted. You will find that solution in the TL;DR section below.

If you would want to keep the old RC_Simple_Color Shader and adjust it to support YUV, you would need to add the second _UVTex Texture as well as the different color conversion versions for YUV. Also, when using the adjusted shader, you would need to set the right ShaderKeywords to tell the shader / material which color conversion to use. Rilissimo already noted that as well. This is done with Shader.EnableKeyword to set the keyword globally for the whole shader and with Material.EnableKeyword to set the keyword for the material only. The hard part is to find out what keywords to use. I couldn't find any function in the Vuforia API to obtain that information, but you could use Material.shaderKeywords on the VideoBackground Material to get the right shader keywords.

TL;DR

In RegionCapture.cs replace line 83:

GetComponent<MeshRenderer>().material.mainTexture = VideoBackgroundTexure;

with:

GetComponent<MeshRenderer>().material = BackgroundPlane.GetComponent<MeshRenderer>().material;

I hope this helps some of you guys. Let me know if there are any issues with this solution. I won't promise to fix anything, but I'd like to know of course. ;)

And thanks again to Rilissimo!

Cheers,

Chris

AttachmentSize
Image icon Screenshot 2019-06-05 at 18.42.42.png177.15 KB

Region Capture

June 6, 2019 - 6:19am #31

Quote:

This is done with Shader.EnableKeyword to set the keyword globally for the whole shader and with Material.EnableKeyword to set the keyword for the material only. The hard part is to find out what keywords to use. I couldn't find any function in the Vuforia API to obtain that information, but you could use Material.shaderKeywords on the VideoBackground Material to get the right shader keywords.





Do not do this if you plan on starting/stopping vuforia

https://developer.vuforia.com/forum/unity/bug-accessing-background-plane-texture-bug

 

And workaround

https://developer.vuforia.com/forum/unity/suggested-workaround-when-accessing-unity-materials-interferes-background-texture-updates



There is a way to get the PIXEL_FORMAT but I can't remember how I did it (and have since scrapped that solution because of the bug above so I can't find it now)

It should have something to do with how you setup the camera, but I could be wrong.



Edit: for search purposes PIXEL_FORMAT is an enum found in the Vuforia.Image class, the first thing that comes up in a web search is here

https://library.vuforia.com/content/vuforia-library/en/articles/Solution/How-To-Migrate-a-Unity-Project.html



Search for pixel_format in there, it might give you a clue

Region Capture

June 7, 2019 - 2:58am #30

Hi guys.

1. Thank you all for your help.

2. For the new Vuforia 8. * you need to use: a) VideoBackground material instead of texture, as in the response of Rilissimo. b) use Renderer.sharedMaterial instead of Renderer.material to fix the problem with backgroundImage hanging on Android.

3. I refactored the source Region Capture plugin, but basically, everything is fine.

Region Capture

June 11, 2019 - 7:25pm #29

I hope they update or fix all bug on region capture for vuforia 8.1 

Region Capture

July 11, 2019 - 3:07pm #28

There's another bug on Android caused by the VideoBackgroundTexure reference in Region_Capture.

To reproduce: on Android do the following:

  1. Start your App, initialize Vuforia
  2. Scan one of your image targets with Region_Capture
  3. Pause the App, e.g. by clicking the app overview button
  4. Resume the App by clicking on it in the app overview

Result: the background video texture freezes

Expected: the background video texture doesn't freeze and continues to show the camera image

Solution/Workaround:

Since we don't need the VideoBackgroundTexure reference any more when using the SharedMaterial directly (like described in the previous posts), make sure to comment out all occurences of VideoBackgroundTexure in RegionCapture.cs and RegionCaptureEditor.cs.

Cheers,

Chris

Region Capture

July 12, 2019 - 6:21am #27

kpprt wrote:

There's another bug on Android caused by the VideoBackgroundTexure reference in Region_Capture.

To reproduce: on Android do the following:

  1. Start your App, initialize Vuforia
  2. Scan one of your image targets with Region_Capture
  3. Pause the App, e.g. by clicking the app overview button
  4. Resume the App by clicking on it in the app overview

Result: the background video texture freezes

Expected: the background video texture doesn't freeze and continues to show the camera image

Solution/Workaround:

Since we don't need the VideoBackgroundTexure reference any more when using the SharedMaterial directly (like described in the previous posts), make sure to comment out all occurences of VideoBackgroundTexure in RegionCapture.cs and RegionCaptureEditor.cs.

Cheers,

Chris

Hello,

Your approach didn`t work for me. I use Unity 2018.4.3 Vuforia 8.1.10

I delete all reference VideoBackgroundTexure. Do you know any solutions?

Region Capture

July 22, 2019 - 4:02am #26

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

Region Capture

August 7, 2019 - 7:24pm #25

klelvin wrote:

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

 

Hello all, 

Same with klelvin, im also facing the same problem. Could someone please let me know whether there is any update on how to fix this problem. Here is the image on this issue.

 

Thank you

Region Capture

August 8, 2019 - 6:50am #24

We had the same issue and had to use an older version of vuforia, however, due to new requirements from Google for 64 bit builds, we had to update Vuforia and the coloring plugin stopped working on the new Vuforia. We haven't figured it out as to why it only effects Android in the new version of Vuforia and we've been picking away at it to figure out the difference. Unfortunately, the plugin is not officially support by Vuforia and the guy who made it did so in his free time, so we may never see an update from him again on it. 

Region Capture - Fix?

September 24, 2019 - 8:09pm #23

Hi there, we are running into the same issue and I was wondering if you ever found a fix or work around? If so, I would really appreciate some support. If it is a bigger undertaking, I'd be happy to pay you for your time. 

 

Thanks!

Region Capture

October 4, 2019 - 5:33am #22

nadzim96 wrote:

klelvin wrote:

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

 

Hello all, 

Same with klelvin, im also facing the same problem. Could someone please let me know whether there is any update on how to fix this problem. Here is the image on this issue.

 

Thank you

 

I have same issue. can someone help now?

Region Capture

October 9, 2019 - 1:09pm #21

Hello everyone. New version 2.2.0 (beta) on GitHub

Regards, Maxim Rouf

Region Capture

October 16, 2019 - 7:44am #20

Hi maximrouf,

 

I`m Using Unity 2019.1.14 and Vuforia 8.3.8. I downloaded the new version of Region Capture on GitHub and open an example scene.

 

When I build the App to an Android device, the texture remains red as before.

 

Do you have any idea what the problem is?

 

 

Region Capture

October 16, 2019 - 11:13am #19

unity 2019.2.8 vuforia 8.3.8 build on android (samsung a30)

 

AttachmentSize
Image icon photo_2019-10-16_21-13-02.jpg40.75 KB

Region Capture

October 23, 2019 - 2:45am #18

gokhany wrote:

nadzim96 wrote:

klelvin wrote:

Hi All, 

Been following this thread for the last couple of weeks, but I'm a bit confused on which issues have been resolved.

The only issue I'm having (after some work) is that the region that is captured is displaying offset on the model - only on Android.  Works fine in the game editor window and on iOS.

- Unity 2019 - Vuforia 8.3.8.

If this has been resolved, could someone please let me know - as I can't seem to find it in this thread for certain, if it hasn't that's OK but I just need to know for sure before I report back to the people who built us an app using older versions about why we can't upgrade.

Thanks

Owen

 

Hello all, 

Same with klelvin, im also facing the same problem. Could someone please let me know whether there is any update on how to fix this problem. Here is the image on this issue.

 

Thank you

 

I have same issue. can someone help now?

 

CAN SOMEONE HELP ME?

Region Capture

October 25, 2019 - 3:23am #17

Hello Everyone,

I'm using :

- Unity 2018.4.11f1 (LTS)

- Vuforia 8.3.8

- Region Capture 2.0.9

I use an Image Target to make my 3D models appears on it. When i tracking the marker all working fine, but when i leave the app and return on it, the camera freeze and all 3D models "flying".

Have you got some idea, how to avoid this ???



Thank you so much guys !

Region Capture

October 28, 2019 - 8:37am #16

- Unity 2019.2.5f1

- Vuforia 8-5-8

- Region Capture 2.2.1

- Device: Xiaomi mi A2 Lite

- Android

- Both (Portrait and landscape)

I have an issue about material. My duck looks red.

Here is a screenshot from my app: https://ibb.co/tmbtMjn

Region Capture

October 31, 2019 - 9:07pm #15

- Unity 2019.2.8f1

- Vuforia 8.3.9

- Region Capture 2.2.1

- Android

Render Camera not capturing region of my marker in android. In editor it's perfectly captured but when i tried on device looks like this https://ibb.co/vcxxyMJ

The raw image on left bottom screen showing image from RenderTextureCamera.GetRenderTexture()

I placed the RegionCapture prefab exactly same as the marker position and scale.

AttachmentSize
Image icon Screenshot_20191101-104736.png2.21 MB

Region Capture

November 5, 2019 - 6:49am #14

Hello everyone. New version 2.2.2 (Final) on GitHub

Regards, Maxim Rouf

Region Capture

November 28, 2019 - 4:00pm #13

Hello there, i tried to develop an AR coloring book. everything works fine, but after some check, try and error, testing, there is some issue i get which is during changing app and try to use back the coloring AR, the camera start to revert back to flip x or rotate 90. So if u can do a favor and help me figure out the problem, it would be much helpful.

Thank you for read this question. All the best

Regard,

Airil

Region Capture

December 11, 2019 - 1:08pm #12

maximrouf wrote:

Hello everyone. New version 2.2.2 (Final) on GitHub

Regards, Maxim Rouf

Unity2019,2,12 and 2,2,5 plugin. I add CameraOutputTexture from RenderTextureCamera.cs to RAW Image in UI. On Unity Editor region cupture works very well and my coloring app are work. Video -> https://youtu.be/JXcP41MrRok

But when i build on Android i have big problems. My Raw Image too bad and coloring app doesnt work(( Video -> https://youtu.be/tt_hFBEAeaA 

I Think its problem because of  FOW AR Camera.

Region Capture

April 16, 2020 - 3:35am #11

Hi,

Can anyone confirm the latest version 2.5 works for Android 64 bit? Our app needs update and we experienced red camera or rotation issues with older version around 1 year ago.

 

Best regards

Region Capture

April 25, 2020 - 1:08pm #10

gokhany wrote:

CAN SOMEONE HELP ME?

for me it has to do with the aspect ratio of the screen of my mobile. You can recreate it in Edit Mode when you use the same screen resolution/aspect ratio.

Region Capture

May 24, 2020 - 3:54am #9

I can't get it working with different screen aspect ratios. 4:3 is the only that works for me. For all other ratios, the captured image is strongly distorted :(

Is this a bug?

EDIT:

Okay so I had a closer look on that and I managed to fix the distorted image on Android.

The problem is that the aspect ratio of your camera video feed is another than the ratio of your screen. For me it's 2240x1080 for the screen and 4:3 for the video feed. If you give the region capture the ARCamera to use WorldToViewport, you get a distored image on your region capture plane. The ARCamera zooms in on the video feed in order to fill the whole screen of the phone.

The solution is to create another camera, on runtime set its position and rotation to the same as of the ARCamera, give it the correct field of view and aspect ratio so that it fits your video feed, and then wire this camera to the region capture. Side note: I don't really know how you can calculate the field of view it, but I found mine out using the ARCamera temporarily setting the aspect ratio of the editor to the one from the video feed and copying the fov of the ARCamera which is set automatically by Vuforia.

Hope it helps,

Regards

Region Capture

May 25, 2020 - 2:01am #8

Hey, could anyone help me. I'm using unity 2019.3.14f, vuforia 9.1.7 and region capture 2.2.5. I already imported all those on unity. But I couldn't find AR camera as shown in this video. How do i find that? This is my final year project. Hoping that anyone could really help me. Thank you! 

https://youtu.be/54W5hF-ZnO8

Region Capture

June 1, 2020 - 1:16am #7

Hey all ,

Can anyone will explain the working of region capture ? I tried to read the code but understood it partially i got basic understanding of some functions in "Region Capture Script"  but i didn't got any intuition  of what is the control flow of program .. can anybody help me out ? I just want to get the basic workflow of how the script is retrieving the image target at runtime. 

Region Capture

October 22, 2020 - 5:04am #6

Hello All,

iOS supports Region Capture or not?

Region Capture

October 22, 2020 - 11:22pm #5

Vuforia 9.5.3 is compatible with Region Capture?

Region Capture

October 22, 2020 - 11:34pm #4

Hy, You got the solution?

Region Capture

April 12, 2021 - 12:00pm #3

i have the same problams

RegionCapture_2.2.5

that work in project very good but in phone don't work (the texture change)

please help me

Region Capture

April 26, 2021 - 7:51am #2

SaPropper wrote:

I can't get it working with different screen aspect ratios. 4:3 is the only that works for me. For all other ratios, the captured image is strongly distorted :(

Is this a bug?

EDIT:

Okay so I had a closer look on that and I managed to fix the distorted image on Android.

The problem is that the aspect ratio of your camera video feed is another than the ratio of your screen. For me it's 2240x1080 for the screen and 4:3 for the video feed. If you give the region capture the ARCamera to use WorldToViewport, you get a distored image on your region capture plane. The ARCamera zooms in on the video feed in order to fill the whole screen of the phone.

The solution is to create another camera, on runtime set its position and rotation to the same as of the ARCamera, give it the correct field of view and aspect ratio so that it fits your video feed, and then wire this camera to the region capture. Side note: I don't really know how you can calculate the field of view it, but I found mine out using the ARCamera temporarily setting the aspect ratio of the editor to the one from the video feed and copying the fov of the ARCamera which is set automatically by Vuforia.

Hope it helps,

Regards

 

I applied  ArCamera postion and rotation at runtime to new Camera. Also , I applied the BackgroudPlane Aspect ratio to the New camera and used it in region capture. But still the issue is not fixed. 

Can you share the piece of code or idea how to fix in Vuforia  9.8  and unity 2019.4.24f.

Region Capture ARFoundation

August 16, 2021 - 12:38pm #1

I think this place is not for ARFoundation but sorry do not have option. I am using ARFoundation but its not working for me. Can you please guide me how I can use this plugin with ARFoundation also If I can get a sample project that also help.

Thanks

Pages

Log in or register to post comments