Hi, I would like to ask if anyone has experience using AR with Cesium. What I am trying to do is to overlay my virtual map on the reality world. I have tried the Magic Leap 2 Tutorial and got it kinda working. What I want to ask is, should I be setting my player to move using AR or using the geo location (lat & long) and disable the AR movement? Also, how can I get the rotation of the virtual map to match the compass?
Hi Wong,
Thanks for posting this use case!
For the smoothest experience you would probably want to position the user once and then rely on magic leap’s movements. With geospatial precision being throttled in various locations, it would be difficult to diagnose a users location at the right precision, however you would to occasionally sync the users root position. I could imagine that you
For the user’s viewing direction, implementing an alignment step might be necessary. I’d be curious to know how much precision from a GPS system you’d have, and what other calibrations you could incorporate given your project goals.
Looking forward to hearing from you,
Ben
Hi Ben,
Thank you for getting back. Currently I have tested using a RTK kit from SimpleArdu, however, it only works outdoor, so I dont find it as much of an impact as compared to the smartphone GPS because at the end of the day, I still need the user to calibrate to overlay it properly.
I am not actually using a Magic Leap 2, I am actually just following the tutorial to get the AR part working for Cesium on Android smartphone. I tried setting the lat & long as the player moves in the real world for both the RTK & Smartphone GPS, no matter what, the overlay will be off when I start moving, so I had a function to lock the lat/long of the player. I haven’t really tried the AR positioning, I will try and let you know again. As for the viewing direction, what do you mean by alignment step? Currently, I am solely using a smooth interpolated compass value. The RTK kit seems to be accurate, if it can help align the direction properly, we can continue to use it.
Another thing I notice is that the generated cesium map is sometimes at an angle, I think possibly due to AR, haven’t looked into how to fix it, so I just restart the app every time it happens. Maybe you know what is going on?
Hi Wong,
Another thing I notice is that the generated cesium map is sometimes at an angle
Would you be able to share an image of this? It’s hard for me to know what that looks like.
In terms of determining the orientation of the virtual environment to the real environment, the biggest bottleneck is the hardware and it’s ability to sense users position and orientation, then map that to the correct virtual position. In terms of the 3D models in Cesium, the alignment to the real world would depend on the source data’s accuracy. Since you’re rendering something on the ground level, comparing a first person view to a global tile set that was created with satellite imagery might contain discrepancies of a few meters. However if you’re working from photogrammetry scans you might get better results.
Any other info would help, thank you!
Hi Ben, apologize for the delayed response and thank you for replying. I assume my locally imported 3D tiles is also using satellite imagery because it overlays nicely with the 3D tiles supplied by Google. Am I right regarding this?
Also, here’s a image example in the editor to simulate what I was seeing. Basically my camera is facing straight, but my whole entire tile map is rotated on the Z-axis. But I do believe it has something to do with AR. I also noticed the map drifting by abit after I manually realign, which most likely due to AR as well. Unless, the Cesium Tileset is moving by itself somehow? What do you think?
Thanks for the image @Wong ! Is the georeference near the user in this image? Have you implemented the Globe Anchor Component?
If not it’s possible you’re experiencing the curvature of the Earth, assuming your georeference is set to a location very far from what’s in the picture. If you want your player gameObject to be rotated such that the feat are on the ground and your oriented upwards, you would need to nest XR Origin inside a gameObject with a CesiumGlobeAnchor component. The globe anchor handles the first rotation of where the user is on Earth, then XR could rotate as needed from the view.
Another approach overall would be to keep the georeference close to the user, by re-centering it at various intervals across the map. This would be a good practice to do occasionally through out the apps runtime, however the most accurate approach is the one listed above.
Outside of that it could be an XR calibration issue.