we are developing an Android application in Unity 2019.2.1f1 using Vuforia 8.3.8 to determine position and orientation of hand-written text in space. For this purpose we created a frame-shaped Image-Target. The written content is placed inside of the marker. Using the RegionCapture plugin (https://github.com/maximrouf/RegionCapture), we record the written content for further processing with OpenCV. At the end the results are pixel-wise compared with a given texture. Therefore the recorded texture should be quite accurate.
However tests revealed a slight distortion and shift of the content on the recorded texture, depending on the viewing angle of the mobile device.This definitely occurs before it reaches the OpenCV processing step. We analyzed this effect and it seems to be of an elliptical shape which let us think that it has something to do with the lense distortion of the camera. You can find a sample image attached. The black pencil strokes are the written content and the light lines behind are the recognized texture.
We took some research how to adapt the camera calibration in Vuforia. Unfortunately Vuforia seems not to support the change of the camera intrinsic parameters as described here: https://developer.vuforia.com/forum/general-discussion/how-vuforia-assign-camera-intrinsic-parameter-across-many-different-camera-devices.
Our next idea was to use a shader to undistort the camera image later on using calibration data obtained e.g. by OpenCV or Vuforia. However then we found https://developer.vuforia.com/forum/qcar-api/camera-calibration-problem, which means the camera image provided by Vuforia should already be undistorted, at least partially. Therefore it's not possible to use standard calibration data.
That's where we are stuck right now. Does anybody have an idea how to fix our problem?