Developing an AR app with the use of Cesium for Unity

Cesium for Unity
1

Hello great forum!

We are a study group from Aalborg University, Denmark. We are currently working on an AR App for a project, where we aim to be able to render a specific cadastre in AR. We have made many iterations of the project and have landed on Cesium for Unity, to perform the georeferencing.

The reason why we post the following questions is to hopefully get some clarification in identifying and solving our issues.

So, to help provide context I will try to list what is working and how it is done as best as possible.

We successfully made a JSON parser that takes the WGS84 coordinates of the points in a specific cadastre, rendered it to show the boundaries. It works! We then take a GNSS to improve the phones’ location to have a precise origin in the world (MockLocation).

The main and headache provoking issue is the georeferencing capabilities, because we can correctly render the cadastre geometry, but whenever we do so in the app it is drawing from the origin set in Unity. And for some reason we cannot figure out how to make the phone the origin when the app is running. We need to render the cadastre from the point where the phone is located with the right orientation.

The last iteration we tried were to make the origin in Unity a known position. We then went outside to the known location to test the app and found that the cadastre geometry is drawn as intended but the orientation of the phone decided where the lines were drawn. The absolute location of the cadastre is relative to the orientation of the phone, but that is the issue, we need the cadastre to be drawn at the right place regardless of the current orientation. The issue correlates with the origin being made in Unity as the place of origin, but we need it to be the phones origin. We can see what the issue is, but we don’t have the knowledge to fix it, and hope to find a solution here. We do not necessarily see an issue with the georeferenced being a Unity unit distance to the points, as long as we can get origin to be correct in relation to the phone location.

If you have any questions or need any clarification, please ask and we will try to provide. The structure of the app in unity is that we have an AR Geospatial Creator Origin with a Cesium Ion tileset as a child. Outside we have CesiumGeoreference GameObject that gets parsed coordinates into a CesiumGlobeAnchor children from the JSON file. These anchors then get connected using a LineRenderer two times, both with fixed height and one being a lower and one being upper, they are then connected using a plane to draw the borders.

Do we need to restructure? Do we need to use OriginShift? Can we make a new origin in the app, after build?

Kind regards,
Andreas, Frederik, Magnus, Niclas and Nicklas.

2

Hey forum,

Is it possible to get a response to our question? Any feedback or advice would be greatly appreciated.

Best regards,
Andreas

3

Hi, @NicklasAAU and @Andreas_Moberg, welcome to the community!
Sorry for the delay, but we don’t have an easy answer for you. @azrogers is going to run through the ARCore Geospatial Creator tutorial when she gets a chance, which will hopefully enable us to offer some better advice. In the meantime, if it’s possible to share a minimal project to reproduce the problem you’re seeing, that will help us help you more quickly.

4

Hello Kevin
Thank you for your answer, it is much appreciated that you put this much effort into helping!

We will provide anything you need, however the project as a whole is difficult to provide, but we can send the whole thing through other channels, if most necessary. Below I have pasted a snippet of the main issue, being the georeference on the go.

public void SetupGeoreference(double phoneLatitude, double phoneLongitude, double phoneAltitude, Quaternion phoneOrientation)
    {
        // If we don't have a valid location yet, use the provided parameters
        double latitude = hasValidLocation ? lastKnownLocation.x : 9.93135578;
        double longitude = hasValidLocation ? lastKnownLocation.y : 57.04874401;
        
        if (georeferenceObject == null)
        {
            georeferenceObject = new GameObject("ManualCesiumGeoreference");
            cesiumGeoreference = georeferenceObject.AddComponent<CesiumGeoreference>();

            if (cesiumGeoreference == null)
            {
                Debug.LogError("Failed to add CesiumGeoreference component");
                return;
            }

            // Set the initial position using the last known location
            cesiumGeoreference.SetOriginLongitudeLatitudeHeight(longitude, latitude, phoneAltitude);
            //cesiumGeoreference.SetOriginLongitudeLatitudeHeight(phoneLongitude, phoneLatitude, phoneAltitude);
        }

        georeferenceObject.transform.rotation = phoneOrientation;
    }

So to take you through our thought process we first setup a CesiumGeoreference GameObject at the phone location. If the phone has not given a location we have a failsafe at a fixed position close to the University.

Then “ManualCesiumGeoreference” gets added and the rendering script finds this GameObject and spawn CesiumGlobeAnchors (this happens when a button is pressed and a JSON file gets parsed with the Latitude / Longitude of the geometry) and draws lines between them.

The whole spawning and drawing works fine, but the way the CesiumGlobeAnchors work by default is calculating the distance from the CesiumGeoreference object to the other location, which is fine. But in our case we can not have the CesiumGeoreference fixed, because in the use case of the app it should be able to draw any cadastre in Denmark. And the behavior at the moment it puts draws the geometry from the phone (0,0,0) but does the distance calculation from the CesiumGeoreference. Which puts the drawn object wrong places. So in other words, the phone needs to be the origin at time of drawing.

The complexity erupts because we get the phone location by two different methods, either just the phone GPS or a GNSS antenna using MockLocation. But the ManualCesiumGeoreference should not care, given the code provided it should just get the Lat/Lon regardless.

It is very hard to put into words, but we hope that this gives some clarification. We hope to either have the CesiumGeoreference update its origin every time the script gets a new location or to spawn one in when the geometry is getting drawn (when the button is pressed). In that way, by giving the precise location and orientation the app will be able to draw the geometry on the location it is placed in the world.

Again, if you need anything, we are happy to provide.

Best regards,
Andreas, Frederik, Magnus, Niclas and Nicklas

5

Based on what you’ve written here, I think your main issue is likely the rotation of the CesiumGeoreference. The Georeference rotation doesn’t update the positions that CesiumGlobeAnchors will calculate from it; it only changes the rotation of any of its children (such as Cesium3DTilesets). Moreover, you shouldn’t need to change the rotation of the georeference based on the rotation of the phone - this should already be handled by the AR Pose Driver component attached to the AR Camera.

So I believe what’s happening here is:

  1. You rotate the phone.
  2. The AR Camera’s rotation is updated based on the rotation of the phone.
  3. The CesiumGeoreference’s rotation is updated, causing the tileset child to rotate. This essentially doubles the phone’s rotation, as it was already applied by the rotation of the camera.
  4. The CesiumGlobeAnchors remain at the position they were previously, as their position is unrelated to the rotation of the CesiumGeoreference. As the Cesium3DTileset was rotated from the rotation of its parent, this causes them to be in the wrong position.

This is the best guess I can give on your problem based on experimenting with the AR Geospatial Creator sample provided by Google. But from previous experience with the Unity AR tools, it can be quite finnicky with how exactly you set your scene up, so it’s possible there’s another issue going on here that I haven’t been able to glean from the information you’ve provided.

If you’re still having issues, would you mind sending me more information, either by private message on here or over email at ashley.rogers@cesium.com? The whole project would certainly help me out, but even just a screenshot of the scene hierarchy would give me a lot more information to work with.

Thank you for the question and I hope this helps you out!