Log in or register to post comments

'ALIGNING an AR view of a room with the ACTUAL view'

October 17, 2015 - 8:06pm #1

Hi,

It has been 3 weeks [including 2 all-nighters] that I have been working on this project on a mini-iPad:

'ALIGNING an AR view of a room with the ACTUAL view'

[as seen via the device camera view in vuforia]

I have tried creating huge markers [1m height] and placing them a short distance apart [multi-marker setup], so that I can get the AR view to align as closely as possible to the actual view, but there is a variance between the farthest points in the AR view and the actual room view.

I have looked at a number of discussions in the developer portal.

Some of the discussions which are closest to what I am looking for are:

https://developer.vuforia.com/forum/unity-3-extension-technical-discussion/ar-camera-field-view

https://developer.vuforia.com/forum/unity-3-extension-technical-discussion/vertical-fov-unity

http://docs.unity3d.com/ScriptReference/Camera-projectionMatrix.html

http://kwazylabs.com/blog/unity_sync_ar_camera_with_2nd_camera/

https://en.wikibooks.org/wiki/Cg_Programming/Unity/Projection_for_Virtual_Reality

I can keep trying all the ideas suggested in the discussions but I am lossing time and there is immense pressure on me to complete my PhD research ASAP [already 3.5 yrs have passed].

I shall therefore be most grateful if someone could kindly commit some of their spare time with me so that I can get through this challenge in the optimum way and in an optimum timeframe.

that sync command is hoax

November 13, 2017 - 11:36pm #11

Simply mentioning that code snippet is not suitable - we need declaration part also to test the validity of the code!

'ALIGNING an AR view of a room with the ACTUAL view'

October 20, 2015 - 3:39pm #10

sounds good, thnx very much!

i did think of the 'trick' which you have mentioned [scaling the model until it fits, i.e. only in the depth direction], and another idea i have thought of is to also create a series of models, so that the display from each marker varies slightly to compensate for the 'far' effect discrepancies....

thanks again!

'ALIGNING an AR view of a room with the ACTUAL view'

October 20, 2015 - 5:43am #9

I would not recommend trying to tweak the projection matrix, cause Vuforia handles that and already tries to produce the most accurate projection matrix based on the knowledge of the camera parameters and the tracking information.

If the augmentation does not fit perfectly  the real world for objects that are far from the center of the marker, as you describe, this is most likely due to the magnification  of the (unavoidable to some degree) precision errors that become more visible as you get farther away from the origin of the tracking frame of reference.

The accuracy may be improved by ensuring a good camera focus, but if you alreay tried that, and still you see some gap to reach the desired result, another thing you could try is to set the Camera Mode to MODE_OPTIMIZE_QUALITY (instead of MODE_DEFAULT).

Ultimately, if you see the augmentation still a bit "small" compared to the real world, rather then tweaking the projection matrix I would probably suggest some simpler "trick" such as slightly increasing the augmentation scale in the Unity scene.

 

 

'ALIGNING an AR view of a room with the ACTUAL view'

October 19, 2015 - 1:06pm #8

i also want to know if it is possible to adjust the perspective projection of the AR view, i.e. via the 'projection matrix' or any other label...

i am asking this because i am trying to display the interior of a room in the AR view and i want to align perfectly with the actual dimensions of that room, but it doesn't...

AR projection starts very accurately [i.e. where it is closest to any of the markers], but the portion of the AR display that is further away from any markers appears shorter on the screen than the actual dimensions of the room...

for example, the AR view shows that the end wall of the room at a certain location, but the actual location of the end wall is a bit beyond that....

i wish i could adjust/tweak the AR projection so it aligns perfectly with the actual dimensions of the room instead of ending a little bit short of the actual dimensions...

thnx in advance!

'ALIGNING an AR view of a room with the ACTUAL view'

October 19, 2015 - 12:54pm #7

thnx, will do that asap!

'ALIGNING an AR view of a room with the ACTUAL view'

October 19, 2015 - 12:36pm #6

Looking at the video, the jump seems due to some inaccuracy in the measurement of the relative size and/or position of the 2 markers with respect to each other;

for example, the size and relative distance (and also the relative orientation angles) in the real world might be slightly different from the ones configured in your Dataset XML and/or in the Vuforia scene.

As an example, you can take a look at the Multi Targets sample; in there, you can see that if you print the Flakes Box, Vuforia will augment it seamlessly (i.e. without positional "jumps" of the augmentation) when the tracked face changes (e.g. when you rotate the box so that it shows the top face instead of the front, or the left face instead of the top one, and so on).

However, in the same example, you could see a jump effect by modifying slightly the target size or the relative positions of the cube faces (e.g. by changing them in the FlakesBox.xml file), i.e. by breaking the gemetric consistency between the "real" target and the ones defined in the XML metadata.

As an additional tip, I would probably enable the continuous autofocus mode at app start, just to be sure that the camera can adjust the focus on the target in view (which may also affect accuracy of the tracking):

https://developer.vuforia.com/library/articles/Solution/Camera-Focus-Modes

 

'ALIGNING an AR view of a room with the ACTUAL view'

October 18, 2015 - 4:48pm #5

i wonder if anyone can tell if the below patch is implementable, or is it a hoax?

doesn't look like a hoax but i can't really figure out how to implement it....

e.g. does Ui camera mean the device camera? if so, then whre in the code can we implement this sync?

It seems that Vuforia’s AR camera uses its own specific version of projection matrix, which defines how the “world” will be shown on screen. A simple script solved the issue, just by applying the AR camera’s matrix to the UI camera:

private bool sync = false;
void Update()
{
    if (!sync)
    {
        uiCamera.projectionMatrix = arCamera.projectionMatrix;
        sync = true;
    }
}

Only after I applied this change – the two cameras got synced. Now I could use the AR camera to detect a target image in the real world, attach a 3D object to that image, then move the object to be the child of the UI camera without experiencing any jump cuts. This makes the object’s new coordinates relate to the app’s UI, therefore I can move it around – to the center of the screen, or anywhere else I want in the app’s UI layout.

any help please?

'ALIGNING an AR view of a room with the ACTUAL view'

October 18, 2015 - 12:42pm #4

this is the subject i am essentially dealing with: http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.html

i am basically trying to find a way to align the AR view perspective projection exactly with the camera view perspective projection....

wish i could find some way of callibrating/syncing these two projections...sort of like the way shown in the attached pic...

i am an architect by profession not an IT whiz, otherwise it would probably be a matter of few minutes i think...

AttachmentSize
Image icon 2.jpg122.59 KB

'ALIGNING an AR view of a room with the ACTUAL view'

October 18, 2015 - 12:15pm #3

here are the results of using a multi-target configuration with extended tracking turned ON:

 - i was trying to check/understand the programmed behaviour of the client if it detects two markers in a multi-marker setup

 - i was expecting it to either render the display object with reference to one marker at a time

 - OR it may try to render the display object with reference to both markers, in which case the dispaly would change according to marker positions

so I set up a simplay display object, a 1m x 1m cube, with two image markers with height 1m placed along/attached to the two far sides of the cube

i am attaching pics of what this looked like in the unity scene view

and then what it looked like in the vuforia view, i.e. exactly the same....

then i did my test manipulation: i moved the markers further apart [next pic]

and the result was that at some instances the display cube jumped from attachment with one marker at a time, as per which ever one it was detecting more strongly

[video of this can be seen at this link: https://drive.google.com/file/d/0B1W8dDSQl78AVUp0XzNkSmpWeGs/view?usp=sharing]

then ... EUREKA .... for a few seconds it displayed the cube as attached to both the markers at once, i.e. at a larger size [in accordance to the increased distance between the two markers], but sadly in a tilted position...   [ pic is attached ]

now i am going to begin testing the ideas given in the documentation on projection matrix adjustments: https://developer.vuforia.com/library/articles/Solution/How-To-Use-the-Camera-Projection-Matrix

i am having trouble understanding exactly what some of the concepts mean, but will keep testing and trying...

any help would be greatly appreciated...

 

 

 

 

 

 

 

 

 

AttachmentSize
Image icon 2.jpg536.47 KB
Image icon IMG_0001.jpg98.9 KB
Image icon IMG_0006.jpg178.98 KB
Image icon IMG_0011.jpg102.37 KB

'ALIGNING an AR view of a room with the ACTUAL view'

October 17, 2015 - 9:06pm #2

some of the ideas that Ithink are most promising are:

  • changing the FOV setting of the AR camera projection matrix....

I know that this is programmed to automatically adopt the device camera FOV value, but perhaps the issue is not the FOV rather how the FOV value is used to generate the AR view....which connects with the next idea

  • adjusting the AR camera projection matrix ...

so that the AR view generates a projection that exactly aligns with the actual projection of a real room as seen via the device camera view in the vuforia display....however this requires manipulating the QCAR core code which I would not like to touch without expert supervision/assistance...

so I shall be grateful if folks could kindly connect me to the right person/expert on such a manipulation....I am sure it will be just a matter of minutes if I know the exact location in the code and also if I know what aspects to be careful of while conducting such a manipulation

as far as I can tell the location of that code is withing the projection.matrix portion, I have been studying it and have some ideas on how to adjust it but I am not 100% sure...

any help will be greatly APPRECIATED!

 

Log in or register to post comments