Aligning point clouds with Cesium terrain

[inealey]

I’m using UE5’s LiDAR point cloud plugin to render some geospatial data on top of the Cesium World Terrain.

Based on the instructions here: Point Clouds – Cesium, I reproject my data to EPSG:4326 before import. However, I find that to properly align with the Cesium tiles, I need to apply a scale factor of 0.85.

When importing through Cesium ION, this scale factor is applied under the hood, but it is not made obvious to the user that this is happening. The instructions say “Cesium ion reprojects your point cloud to EPSG:4326”, with no mention of any scaling.

Does anyone know the source of this scale mismatch? It’s not a showstopper, but I find it odd that it’s not documented.

Due to privacy concerns and rendering optimizations, “just use Cesium ION to import the point clouds” is not an acceptable solution here.

[Kevin_Ring]

Hi @inealey, welcome to the community!

I’m not quite sure what’s going on here. Cesium ion will preserve the geospatial accuracy of the data you import to the best of its knowledge. Accuracy and precision are at the core of what we do, we wouldn’t intentionally apply an arbitrary scaling factor for no reason.

I’m having a little trouble following what you’re doing. You said you’re using the LiDAR plugin to render some geospatial data on top of Cesium World Terrain. So what data are you importing in Cesium ion? Or are you just importing the same point cloud data into ion as well in order to compare it against the LiDAR plugin (even though you said you’re not able to use ion)?

[inealey]

Thanks Kevin,
and I appreciate the quick response!

Or are you just importing the same point cloud data into ion as well in order to compare it against the LiDAR plugin (even though you said you’re not able to use ion)?

Right, as a sanity check I imported a point cloud into ion just to compare with the LiDAR plugin. Our team unfortunately cannot use ion long-term as some of our data has restrictive licensing.

we wouldn’t intentionally apply an arbitrary scaling factor for no reason

It does not seem arbitrary, as I have observed consistent behavior across multiple data sets from different sources.
It could, however, simply be user error on my part, or some issue in PDAL (which I use for reprojection), or an import bug in the LiDAR plugin from UE5.

Let me generate an example using public dataset and I will share the results/source files here. I would love to get to the bottom of this.

[Kevin_Ring]

Our team unfortunately cannot use ion long-term as some of our data has restrictive licensing.

I’m curious about this, because using Cesium ion in no way implies your data is public. Most of our users have data with restrictive licensing.

We also have a Self Hosted product, if your data absolutely cannot be stored by Cesium:

Cesium ion Self-Hosted

Let me generate an example using public dataset and I will share the results/source files here. I would love to get to the bottom of this.

Ok, let me know how you go.

[inealey]

OK! I was able to dig into this a little bit and came to the same results as before.
Let me document my process and we’ll see how to move forward.

1. For example point cloud data, lets choose an example from California’s beautiful Yosemite Valley. I downloaded USGS_LPC_CA_YosemiteNP_2019_D19_11SKB6680 from the USGS 3DEP dataset (publically available at: USGS Lidar Explorer Map).
   This is an ~85M point LiDAR cloud with a native projection of UTM Zone 11N + NAVD88 height - which correlates to the EPSG code of 6340 + 5703.

2. I used PDAL to decimate the cloud by factor of 10 (with random sampling) to make it easier to work with. This does not affect the projection but just clarifying in the interest of transparency. So now I have a ~8.5M point LiDAR cloud with a CRS of EPSG:6340+5703.

3. Alright, lets upload to ION and download using Cesium for Unreal…
   *some reprojection processing happens*
   ION places the cloud’s center at the same coordinates indicated in the USGS metdata. Great! Although some of the points stick through the terrain, I would not expect the 2019 USGS data to magically align with Cesium’s 2016 terrain.

It “fits” well enough that I believe the coordinate systems are aligned.
It even appears as though the LiDAR trees line up with those seen in the Bing Maps imagery. pretty neat!

4. So, how can we use the LiDAR plugin to render this same dataset? The naive approach would be to simply import the point cloud into Unreal in its native projection. I did that, and positioned it on the Cesium terrain with a CesiumGlobeAnchor component. That looks like this:
   
   
   
   
   
   
   
   

The datasets are slightly misaligned. Red box is the bounds of the Cesium tileset (colored white) and blue box is the bounds of the imported LiDAR point cloud (colored red–>green by elevation).

5. The documentation mentions that ION will reproject your data to EPSG:4326. This may make sense as an intermediate step, but Unreal’s native coordinate system uses centimeters as its unit of measurement, and 4326 uses degrees. As such, importing the point cloud data as EPSG:4326 will not work (I was mistaken about this in my original post, I have been using a different CRS). We need some CRS that uses metric units.

6. Based on this comment (py3dtiles convert gives weird result if output projection format specified · Issue #66 · Oslandia/py3dtiles · GitHub) and this note in the 3D tiles spec (3d-tiles/specification at main · CesiumGS/3d-tiles · GitHub), I tried EPSG:4978 next. This looks similarly close in size to the Cesium tileset, but off by an even more substantial transformation:
   

   

7. Based on some research about map services, I tried web mercator (EPSG:3857) and world mercator (EPSG:3395). Now this is where it gets interesting. These seem to be the closest to the Cesium projection, although they are off by some scale factor near 0.8 in the XY axes.

This is the same place I had come to before with another dataset in EPSG:5070, which had a similar scale factor difference (the “0.85” in my original post).

8. For this specific Yosemite dataset, after projecting to EPSG:3395 then applying a scale factor of ~0.79 in the XY axes, it aligns with the Cesium tile perfectly. For EPSG:3857, The scale factor is asymmetrical, with a slightly lower value in the Y axis.

Here is what EPSG:3395, or world mercator, looks like after importing:

and now after scaling to fit the Cesium tileset:

Looks great!
…but I can’t figure out what the ION processing pipeline is doing that PDAL is not. And I still don’t feel confident that I have the projection correct. Having to transform/scale at all after importing seems like a red flag.

Apologies for the long post…
@Kevin_Ring perhaps you could share some insight about what happens when a user uploads a point cloud to cesium ION? Is it solely projected to a new CRS or is there also some refinement/transformations?
What is the EPSG code of the Cesium terrain tiles CRS?

Regards,
- Isaac

[Kevin_Ring]

Cesium ion produces 3D Tiles in the Earth-Centered, Earth-Fixed (ECEF) coordinate system, which is EPSG:4978, as required by the specification:

3d-tiles/specification at main · CesiumGS/3d-tiles

Cesium for Unreal then maps this into Unreal coordinate system as specified by the CesiumGeoreference Actor. There’s a bit more about how this works here:

Placing Objects on the Globe

I haven’t used the LiDAR plugin much myself, but I can take an educated guess about what it’s doing based on what you’re seeing.

When you give it a point cloud in a projected coordinate system, it appears to be mapping that 1:1 into Unreal coordinates without unprojecting. Imagine your Unreal world were a 2D map (plus height). When you’re zoomed in close, it might not be obvious this is the case, but it would in fact have the usual the distortions in angles and area that are the inevitable result of any map projection. It won’t align with the Cesium world in which Earth is round (as it is in reality, of course).

When you transform your point cloud to EPSG:4978, you’re unprojecting it, to the same ECEF coordinate system that 3D Tiles uses. So far, so good! Except the orientation is all wrong because the LiDAR plugin is not correctly mapping the point cloud into the Unreal coordinate system. In the Unreal world, +Z is up (at least near the CesiumGeoreference). In ECEF, +Z starts at the center of the Earth and goes up through the North pole. The misalignment between the two will depend on where you are on Earth.

Depending on how the EPSG:4978 transformation works, you may also be losing precision at that step. If it truly uses the center of the Earth as the origin of the coordinate system, then the coordinates of each point will be very large.

In any case, you may be able to fix the misalignment by using Unreal’s GeoReferencing plugin to fix this (but I’m not familiar with the details myself). Or you can use Cesium ion, which handles all of this for you, and which directly supports the ongoing development of Cesium for Unreal.

[inealey]

Cool! A couple follow up questions

the orientation is all wrong because the LiDAR plugin is not correctly mapping the point cloud into the Unreal coordinate system

So I just need to reorient the axes, and it should align?

Depending on how the EPSG:4978 transformation works, you may also be losing precision at that step.

What is ION doing during the import process that prevents this loss of precision?

you may be able to fix the misalignment by using Unreal’s [GeoReferencing] plugin to fix this

Just seeing this now, thanks for the tip!

I would prefer this thread to remain on the topic of how to align with Cesium tilesets without ION, but since it has come up again, here is the line from the ION TOS that renders it useless for our purposes:

In order for Cesium to host Your Content and Processed Content, you grant Cesium a non-exclusive, worldwide, royalty-free, transferable, and sub-licensable right and license to use, display, use, cache, copy, distribute, modify, transmit, and store Your Content and Processed Content. You affirm that you have the right to grant these rights to Cesium.

**edit: I realized what you mean by mapping from ECEF to Unreal Units, and why it’s happening this way. I will follow up here after some more digging into the georeferencing plugin and other resources.
Any insight into how ION performs this mapping (i.e. how to get the transformation for a given location on the ellipsoid) would be greatly appreciated.

[Kevin_Ring]

So I just need to reorient the axes, and it should align?

It looks like something - perhaps the LiDAR plugin, or perhaps the reprojection operation you did - has already given the point cloud a local origin. So yes, you just need to fix the orientation.

What is ION doing during the import process that prevents this loss of precision?

The (single-precision) point positions are all relative to a local origin, instead of expressed directly in ECEF. Then, a double-precision transformation matrix maps the model into ECEF.

here is the line from the ION TOS that renders it useless for our purposes:
In order for Cesium to host Your Content and Processed Content, you grant Cesium a non-exclusive, worldwide, royalty-free, transferable, and sub-licensable right and license to use, display, use, cache, copy, distribute, modify, transmit, and store Your Content and Processed Content. You affirm that you have the right to grant these rights to Cesium.

The key part is at the start of that sentence: “In order for Cesium to host Your Content and Processed Content…” We need the right to copy it in order to provide the service. For example, if we had to ask your permission to store your data in another availability zone on AWS, that would severely hinder our ability to provide the service. However, you control access to your data on Cesium ion, and we have no intention to distribute your data to anyone other than as required to operate the service. I don’t believe that clause grants us the right to do so (though I agree it could be worded better). I am of course not a lawyer, but if you have concerns, please reach out to our sales folks to discuss, rather than assuming you can’t use Cesium ion.