Log in or register to post comments

camera patch in unity

August 16, 2014 - 8:49am #7

Hello,

I've been struggling to get the camera patch sample to work in unity.

I finally succeded to make it work in the editor but for the life of me can't make it work on ios. 

The only difference I can see is cameraTextureRatio  which is 1.0 1.0 in the editor the texture and image size being identical.
On my iphone it is: 

_CAMERATEXTURE_RATIO_X:0.625

_CAMERATEXTURE_RATIO_Y:0.9375

with resolutions of:

VideoTextureInfo 1024 512 640 480

 

I'm calculating it like so:

cameraTextureRatio.x = (float)texInfo.imageSize.x / (float)texInfo.textureSize.x ;
cameraTextureRatio.y = (float)texInfo.imageSize.y / (float)texInfo.textureSize.y ;
 
(but I tried swapping x,y, and about every value there is, and set the player to portrait/landscape etc.)
 
 
The shader is really almost identical to the one in the sample:
float4x4 _MATRIX_MVP;
	float _CAMERATEXTURE_RATIO_X = 0;
	float _CAMERATEXTURE_RATIO_Y = 0;

	struct v2f{

		float4  pos : SV_POSITION;
		float2  uv : TEXCOORD0;
	};

	v2f vert(appdata_base v){
		
		v2f o;
		float2 targetSize = float2(10,10);
		float2 cameraTextureRatio = float2 (_CAMERATEXTURE_RATIO_X, _CAMERATEXTURE_RATIO_Y);
		
		float4 targetPoint = float4(targetSize.x * (v.texcoord.x - 0.5), targetSize.y * (v.texcoord.y - 0.5), 0.0, 1.0);
		float4 clipCoords = mul(_MATRIX_MVP, targetPoint);
		float3 normalizedDeviceCoords = clipCoords.xyz / clipCoords.w;
		
		float2 cameraTexCoords = float2((normalizedDeviceCoords.x + 1.0) / 2.0, ((normalizedDeviceCoords.y * -1.0) + 1.0) / 2.0);
        float2 finalCoords = float2(cameraTexCoords.x * cameraTextureRatio.x, cameraTexCoords.y * cameraTextureRatio.y);
        o.uv = finalCoords;
        o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
        
		return o;
		
		
	}

 

There must be something else I'm missing - if anybody has a hint I'd really appreciate it to know.. desperate... :/

 

Thanks a lot in advance,

Seb

 

sschoellhammer
Joined:
August 12, 2014
Posts: 8
Top

camera patch in unity

August 17, 2014 - 9:37am #6

Little update:

so it actually *does* work on the device - but only if I set the camera direction to CAMERA_FRONT. I suppose now the camera is matching the one from the webcam in the unity editor and I get the same result.

I bet there is just one thing that needs to be inverted ..  *somewhere* ..  :/

Any eye-openers, please send them my way.

 

Thanks muchly!

seb

sschoellhammer
Joined:
August 12, 2014
Posts: 8
Top

camera patch in unity

August 18, 2014 - 5:38am #5

If I change 

float2 cameraTexCoords = float2((normalizedDeviceCoords.x + 1.0) / 2.0, (-normalizedDeviceCoords.y + 1.0) / 2.0);

to

float2 cameraTexCoords = float2((-normalizedDeviceCoords.x + 1.0) / 2.0, ( normalizedDeviceCoords.y + 1.0) / 2.0);

it works *better* with the forward camera. 
 
However I now also noticed that it seems to work only *perfectly* if the player aspect ratio is 16:9 . Then the patch is unmovably, beautifully captured. If the ratio is anything else there is some skewing happening. I'm pretty clueless why as I thought it would be part of the projection matrix that gets passed to the shader.
 
 
I'm on the brink of giving up, I think I've tried all there is by now.. :/
 
I've made a clean unity project here:
 
I'd be eternally grateful if somebody could have a look and maybe spot a mistake.
 
the marker used is in Textures/ar.png
 
When run the captured patch gets displayed in front of the camera.
 
Thanks in advance,
 
Seb
 
sschoellhammer
Joined:
August 12, 2014
Posts: 8
Top

camera patch in unity

September 15, 2014 - 3:34am #4

Hi sschoellhammer,

amazing!! I'm test the code and seems that works fine, but, how can I stop the update texture proces when it tracked correctly the marker?

I'm tried to create a boolean in the OnTrackableStateChanged, and the update proces, but don't works.

I would to stop the process once the texture is captured after "1" second is captured a correct marker, I'm tried also the SaveTexture function, but always with not correct frames.

 

Could you help me?

Alvaro

    void Update() {

        if (m_isVideoReady){

            if (!m_textured){
                CopyPatchToTexture();
            }

        }        
    }
        public void OnTrackableStateChanged(

        TrackableBehaviour.Status previousStatus,
        TrackableBehaviour.Status newStatus)
    {

        if (newStatus == TrackableBehaviour.Status.DETECTED ||
            newStatus == TrackableBehaviour.Status.TRACKED ||
            newStatus == TrackableBehaviour.Status.EXTENDED_TRACKED)
        {
            OnTrackingFound();
            m_isTracking = true;

            if (!m_textured){
                Debug.Log("hello");
                //SaveTexture();
                m_textured=true;
            }

        }
        else
        {
            OnTrackingLost();
            m_isTracking = false;
            m_textured=false;
        }
    }

alvaroull
Joined:
November 9, 2011
Posts: 13
Top

camera patch in unity

September 19, 2014 - 1:04am #3

sschoellhammer,

can you explain to me how you make the camera patch work? TQ

jasoncytan
Joined:
September 5, 2014
Posts: 6
Top

camera patch in unity

September 21, 2014 - 10:24am #2

Hi sschoellhammer,

actually you're right, 

if the resolution is 16/9 works perfectly. 
I've been testing different resolutions in several devices and it seems that this mismatch as you say in the mapping shader, but my knowledge does not come to find out why, so if anyone can help us out, we would be very grateful. 
 
Anybody from the development Team? AlessandroB?
 
Alvaro
alvaroull
Joined:
November 9, 2011
Posts: 13
Top

camera patch in unity

October 20, 2014 - 2:29am #1

Hello.

I tried this patch in many devices, in the other(non 16:9 resolution) devices the texture patch gets scaled in x direction. The buffer size is diferent to the texture size.I searched and found it some related to power of two restriction of hardware.

the resolution is when 1280*720 (16:9) the buffer is of 2048*1024.

the ratio 1280/2048 =0.62 and 720/1024=0.70 is somewhat closer with respect to when resolution is 4:3:-

640*480 (4:3) the buffer is of size 1024*512 ans the ratio 640/1024=0.62 and 480/512=0.93. There must be some scaling in x direction which you can observe when you view the plane(RenderTargetView) in your device. I tried to resize the texture but unity wont allow it.

Does anyone have an idea how to resolve this issue? 

abhayshankar
Joined:
August 25, 2014
Posts: 4
Top
Log in or register to post comments