Geolocation as External Prior for Area Targets

This article describes how to use GPS (Global Positioning System) coordinates as External Prior to relocalize large Area Targets.

Applications with Area Targets for scanned areas that are very large require injected External Priors to initialize tracking. In some other scenarios, the Device Tracking over longer distances may get lost, or localization purely based on visual information proves difficult, using the External Priors with geolocation may be the solution.

To get started, please see Area Targets for obtaining your own scan to create an Area Target from and see also the Area Target API Overview -- External Prior for an introduction to the concept of injecting an external position for better environment detection and tracking.

Geopositioning on Mobile Platforms

Originally, GPS-localization has been designed for outdoor, large-scale, use cases and relies on a clear signal between your device and multiple satellites orbiting Earth. Today's "GPS"-receivers use additional Global Navigation Satellite Systems (GNSS) such as GLONASS, BeiDou, Galileo and others -- which all are limited from similar line-of-sight and other radio-problems like the original GPS system. The accuracy of geopositioning significantly drops or is even in some cases impossible in indoor environments due to such issues.

To overcome this, all mobile platform providers have implemented hybrid or fused mobile positioning systems that rely in addition to GNSS on Network Location Provider (NLP) positioning using cellular antennas, Wifi-signals, as well as magnetometer, and other sensors. Such a fused geolocation provides accurate, smooth, ubiquitous, and instant geopositioning for mobile devices.

The device location can be determined by a platform API that consolidates information based on the different sources and provides a fused location in the World Geodetic System -- in its latest revision, the WGS 84 geographic coordinate system. The location consists of latitude, longitude, and elevation -- alongside with other information, such as timestamps and accuracy information.

Area Targets and Geolocation

Vuforia Engine pose-information and Area Targets use a Cartesian reference frame with coordinates in meters (Further information on Vuforia's system coordinates is available on the Spatial Frame of Reference page).

A location prior (external position) from a device geolocation must therefore take the same reference frame before being injected. To compute this transformation, you need to know the geolocation and orientation of the Area Target's origin.

You will need to align the target in the WGS 84 coordinate system -- commonly called georeference. Once this reference has been established, the device pose provided during tracking in the Area Target's coordinate system, and asset- or augmentation locations in WGS 84 can be transformed between the two coordinate systems freely back and forth.

NOTE: In this description and sub-sequent pages we may sometimes refer to the location in the geographic coordinate system WGS 84 simply as "geolocation".

Overall Workflow

The general workflow consists of three stages:

1. Georeference the Area Target: Obtain the location of your Area Target's origin and its rotation angle with respect to the WGS 84 geographic coordinate system. We describe two different options to achieve this:
   1. Align the Area Target in the Unity Editor with a satellite image of the surrounding area to retrieve geolocation and orientation of the origin.
   2. Measure a few points with a custom app, logging both Vuforia Area Target poses and GPS readings on-site. Use the provided Python script to compute the offset of the Area Target origin.
2. Use device geolocation as prior: To kickstart Area Target tracking or aid localization using an external prior, convert during app-runtime the measured GPS-device-location to Area Target Cartesian coordinates. Inject these as an external prior to your Area Target. Do this every time a new GPS position is available.
3. Render georeferenced assets: Optionally, display georeferenced information, such as IoT-sensor data, BIM-created assets, or labels in your augmentation, by transforming their geolocation into the Area Target's Cartesian coordinate system.

Three alternative paths to georeference your Area Target prior to application development is described on the Georeferencing Area Target page. The result of your alignment yields the latitude and longitude of the Area Target's origin, and its orientation with respect to Earth's North Pole. This information will be needed in the next sections below.

NOTE: This guide only shows how to inject the latitude and longitude as external prior.

Use Device Geolocation as Prior

Use the results of the georeferenced Area Target as prior at runtime. To do so, we show:

• How the fused geolocation can be obtained in Unity,
• Which transformations need to be applied.
• What it takes to inject the geolocation as a prior.

Get Device Geolocation in Unity

Read the geolocation of a mobile device using the Unity Input.location API to receive global coordinates for cross-platform apps. Refer to Unity - Scripting API: LocationService for more details.

The location service can be started with:

Input.location.Start(Accuracy, UpdateDistance);

To query the last reported geolocation, use:

var location = Input.location.lastData;

The below script shows the handling of this API checking and continuous coordinate updates, provided that the user has allowed the location service to run. Attach the below GPSLocationProvider script to a new GameObject to obtain the device's geolocation as a debug output. To conserve battery, you can indicate the accuracy requirements and the update-rate when the OS provides updated geolocation information. Based on these settings the most optimal sources and operational mode is chosen by the OS. 

/*========================================================================
 Copyright (c) 2019-2022 PTC Inc. All Rights Reserved.

 Vuforia is a trademark of PTC Inc., registered in the United States and other
 countries.
 =========================================================================*/
using System.Collections;
using UnityEngine;
using UnityEngine.UI;

// Behavior and usage of this class is described in the public article
// "Area Targets with GPS External Position"
namespace Geography
{
    public class GPSLocationProvider : MonoBehaviour
    {
        public bool Initialized { get; private set; }

        public float Latitude { get; private set; }
        public float Longitude { get; private set; }
        public float Altitude { get; private set; }

        [Header("General Settings")]
        [Tooltip("The desired accuracy in meters.")]
        public float Accuracy = 5f;

        [Tooltip("The update distance in meters.")]
        public float UpdateDistance = 5f;

        [Tooltip("Automatically start GPS service on awake")]
        public bool StartOnAwake = true;

        void Awake()
        {
            if (StartOnAwake)
            {
                StartCoroutine(StartLocationServiceAsync());
            }
        }

        void Update()
        {
            if (!Initialized)
                return;

            var lastLocationData = Input.location.lastData;
            float accuracy = lastLocationData.horizontalAccuracy;
            Latitude = lastLocationData.latitude;
            Longitude = lastLocationData.longitude;
            Altitude = lastLocationData.altitude;

            Debug.Log($@"Location:
                 {System.Environment.NewLine} lat: {Latitude}
                 {System.Environment.NewLine} lon: {Longitude}
                 {System.Environment.NewLine} alt: {Altitude}
                 {System.Environment.NewLine} accuracy: {accuracy.ToString("0.0")}");
        }

        public void StartLocationService()
        {
            StartCoroutine(StartLocationServiceAsync());
        }

        IEnumerator StartLocationServiceAsync()
        {
            Debug.Log("Trying to access location...");

            // First, check if user has location service enabled
            if (!Input.location.isEnabledByUser)
            {
                Debug.Log("Location access not enabled by user.");
                yield break;
            }

            // Start service before querying location
            Input.location.Start(Accuracy, UpdateDistance);

            // Wait until service initializes
            const int maxTries = 10;
            int tryCount = 0;
            while (Input.location.status == LocationServiceStatus.Initializing &&
                    tryCount < maxTries)
            {
                yield return new WaitForSeconds(1);
                tryCount++;
            }

            // Service didn't initialize
            if (tryCount >= maxTries)
            {
                Debug.Log("Location service initialization timed out");
                yield break;
            }

            // Connection has failed
            if (Input.location.status == LocationServiceStatus.Failed)
            {
                Debug.Log("Unable to determine device location");
                yield break;
            }

            // Access granted and location value could be retrieved
            if (Input.location.status == LocationServiceStatus.Running)
            {
                Initialized = true;

                var lastLocationData = Input.location.lastData;
                Debug.Log($@"Location:
                     {System.Environment.NewLine} lat: {lastLocationData.latitude}
                     {System.Environment.NewLine} lon: {lastLocationData.longitude}
                     {System.Environment.NewLine} alt: {lastLocationData.altitude}
                     {System.Environment.NewLine} accuracy: {lastLocationData.horizontalAccuracy}
                     {System.Environment.NewLine} time: {lastLocationData.timestamp}");
            }
        }
    }
}

NOTE: You will need to specifically ask for runtime permission for the location services to be used. See Unity's Request User Permission API for more details.

Convert Geolocation GPS-Coordinates to Cartesian Coordinates

It is necessary to convert the geolocation obtained in the previous section to cartesian (XY) coordinates relative to the local reference frame of the Area Target.

To perform this step, it is necessary to know the following information:

• The fused geolocation (Latitude, Longitude) of the origin of the Area Target,
• The orientation angle of one of its main axes with regard to the local geographic North direction.

This data can be obtained by following the Georeferencing Area Target guide. To store the origin values, create a script named GeoLocation.cs.

/*========================================================================
 Copyright (c) 2019-2022 PTC Inc. All Rights Reserved.
 Vuforia is a trademark of PTC Inc., registered in the United States and other
 countries.
 =========================================================================*/

[System.Serializable]
public class GeoLocation
{
    public float latitude;
    public float longitude;
}

Use then the following GPSLocationMapper.cs script with the geographic reference of the Area Target's origin to convert the output values from the user position that is provided at runtime by the GPSLocationProvider.cs script and convert those to cartesian coordinates. Add the GPSLocationMapper.cs to an empty GameObject and attach the necessary objects and parameters to its component in the Inspector.

Once the cartesian (XY) coordinates of a GPS location have been determined in the local reference frame of the Area Target 3D space, it is injected into the SetExternal2DPosition method of the Vuforia Area Targets API:

/*========================================================================
 Copyright (c) 2019-2022 PTC Inc. All Rights Reserved.

 Vuforia is a trademark of PTC Inc., registered in the United States and other
 countries.
 =========================================================================*/
using UnityEngine;
using Vuforia;

// Behavior and usage of this class is described in the public article
// "Area Targets with GPS External Position"
namespace Geography
{
    public class GPSLocationMapper : MonoBehaviour
    {
        [Header("Input data")]
        public GPSLocationProvider GpsLocationProvider;

        [Header("Area data")]
        public AreaTargetBehaviour AreaTarget;

        [Tooltip("The geo-location of the center-point (origin) of the Area Target reference frame.")]
        public GeoLocation AreaCenter;

        [Tooltip("The angle (CCW degrees) of the Area Target Z-vector to the North.")]
        public float OrientationAngle = 0f;

        [Header("Debug UI")]
        public UnityEngine.UI.Text infoText;

        void Update()
        {
            if (!GpsLocationProvider || !GpsLocationProvider.Initialized)
                return;

            var xyzCoords = GetCartesianCoordsFromGPS();

            if (infoText)
                infoText.text = "XYZ coords: " + xyzCoords.ToString("0.0");

            if (AreaTarget)
            {
                AreaTarget.SetExternal2DPosition(
                    position: new Vector2(xyzCoords.x, xyzCoords.z),
                    horizontalAccuracy: 5f
                    );
            }
        }

        /// <summary>
        /// Returns cartesian coordinates in meters in the local Area Target
        /// reference, corresponding to the Lat and Lon provided by the
        /// GPSLocationProvider.
        /// </summary>
        /// <returns></returns>
        Vector3 GetCartesianCoordsFromGPS()
        {
            var userLat = GpsLocationProvider.Latitude;
            var userLon = GpsLocationProvider.Longitude;
            var userAlt = 0.1f;

            var areaCenterLat = AreaCenter.latitude;
            var areaCenterLon = AreaCenter.longitude;
            var areaCenterAlt = 0f;

            // Transform the latitude, longitude and altitude to
            // North-East-aligned cartesian coordinates.
            // Note:
            // +X points East
            // +Y points Up
            // +Z points North
            var positionXYZ = GeoUtil.LatLonToXYZ(
                userLat, userLon, userAlt,
                areaCenterLat, areaCenterLon, areaCenterAlt
                );

            // Transform the cartesian coordinates from North-East-aligned
            // to the local area reference, taking the orientation into account
            var angleRad = OrientationAngle * Mathf.Deg2Rad;
            var xr = positionXYZ.x * Mathf.Cos(angleRad) - positionXYZ.z * Mathf.Sin(angleRad);
            var zr = positionXYZ.x * Mathf.Sin(angleRad) + positionXYZ.z * Mathf.Cos(angleRad);
            positionXYZ = new Vector3(xr, positionXYZ.y, zr);
            return positionXYZ;
        }
    }
}

The above script needs an additional utility script to convert from geographic to cartesian coordinates.

NOTE: For higher accuracy, use System.Math for calculations involving double-precision (64 bit) floating point numbers, as opposed to using UnityEngine.Mathf which operates on single-precision (32bit) floating point numbers.

/*========================================================================
 Copyright (c) 2019-2022 PTC Inc. All Rights Reserved.

 Vuforia is a trademark of PTC Inc., registered in the United States and other
 countries.
 =========================================================================*/
using UnityEngine;

// Behavior and usage of this class is described in the public article
// "Area Targets with GPS External Position"
namespace Geography
{
    public class GeoUtil
    {
        const double EARTH_RADIUS_METERS_NORTH_POLE = 6356752;
        const double EARTH_RADIUS_METERS_EQUATOR = 6378137;
        const double DEG2RAD = 0.01745329252;

        public static Vector3 LatLonToXYZ(
            double latDeg, double lonDeg, double alt,
            double centerLatDeg, double centerLonDeg, double centerAlt)
        {
            double latRad = latDeg * DEG2RAD;
            double relLatDeg = latDeg - centerLatDeg;
            double relLonDeg = lonDeg - centerLonDeg;
            double relAlt = alt - centerAlt;

            double localEarthRadius = GetEarthRadius(latDeg);

            // One degree of latitude is ~110 Km at the equator, precisely:
            double latDeg2Meters = localEarthRadius * DEG2RAD;

            double longitudeRadius = localEarthRadius * System.Math.Cos(latRad);
            double lonDegToMeters = longitudeRadius * DEG2RAD;

            // Note:
            // +X points East
            // +Y points Up
            // +Z points North
            double x = relLonDeg * lonDegToMeters;
            double y = relAlt;
            double z = relLatDeg * latDeg2Meters;
            return new Vector3((float)x, (float)y, (float)z);
        }

        // Computes the local earth radius at the given latitude,
        // by interpolating between the Equator radius and the
        // North Pole radius.
        static double GetEarthRadius(double latitudeDeg)
        {
            double theta = latitudeDeg * DEG2RAD;
            double ra = EARTH_RADIUS_METERS_EQUATOR;
            double rb = EARTH_RADIUS_METERS_NORTH_POLE;
            double rst = ra * System.Math.Sin(theta);
            double rct = rb * System.Math.Cos(theta);
            return ra * rb / System.Math.Sqrt(rst * rst + rct * rct);
        }
    }
}

Learn More

Georeferencing Area Targets

Area Targets API Overview

Area Targets