Transform specified in 3D tile data

Cesium for Unreal
1

Hi all,

In some 3D Tile datasets, there is a transform node. Looking at Cesium Ion’s Melbourne dataset (69380), there is a transform specified in the root tileset.json:

"transform":[-0.5699774740294932,-0.8216313954439031,-0.006894136626272664,0,-0.5068551282833045,0.34498402763987435,0.7899898098116428,0,-0.6467020628015874,0.4537707248096024,-0.6130812109940204,0,-4129944.4597646086,2897853.584712872,-3889026.1577356746,1]

For this dataset (69380), suppose we’re loading 2\010.b3dm. The b3dm does not have any transforms specified, so we’re focused on the 3DTile transform shown above. Is it true that any vertex of 010.b3dm can be transformed using the transform above to get that vertex’s ECEF location?

How do I create an FTransform (with the values from “transform” show above) in UE5 that can be used to transform locations?

Is this correct?

FTransform datasetTransform = FTransform(
	FMatrix(
	FPlane(-0.5699774740294932, -0.5068551282833045, -0.6467020628015874, 0),
	FPlane(-0.8216313954439031, 0.34498402763987435, 0.4537707248096024, 0),
	FPlane(-0.006894136626272664, 0.7899898098116428, -0.6130812109940204, 0),
	FPlane(-4129944.4597646086, 2897853.584712872, -3889026.1577356746, 1)
));

Should the code below give the correct ECEF for a vertex?
vertexECEF = datasetTransform.TransformPosition(vertexLocation);

Thanks!

2

Is it true that any vertex of 010.b3dm can be transformed using the transform above to get that vertex’s ECEF location?

Yes. Assuming what you say is true that the b3dm itself doesn’t have any transforms (I haven’t checked). And assuming that 2/010.b3dm is either attached directly to that root tile, or the “path” of child tiles doesn’t have any additional transformations.

How do I create an FTransform (with the values from “transform” show above) in UE5 that can be used to transform locations?

Your code looks correct to me.

Should the code below give the correct ECEF for a vertex?

It should.

If this isn’t working for you, the most likely cause is that there is an extra transform lurking in the b3dm itself, such as in the RTC_CENTER semantic:

3

I’m definitely getting some odd values for ECEF. The longitude and latitude appear to be correct, but the HAE seems wrong. Here’s an example:

image
image2432×942 273 KB

Note that the HAE (3rd value of coordinate) ranges from -1300m to 1300m. Melbourne is around 10m HAE, so the values I’m getting seem off.

I’m not seeing RTC_Center in the b3dms for the Melbourne dataset. Here’s the header for 2/010.b3dm:

{"BATCH_LENGTH":0} glTF ¨ûa ¤ JSON{"asset":{"version":"2.0"},"accessors":[{"componentType":5123,"type":"SCALAR","count":43629},{"componentType":5126,"type":"VEC3","count":19876,"min":[-4495.369140625,-10.204215049743652,-371.314453125],"max":[-2314.611083984375,90.30708312988281,2300.718505859375]},{"componentType":5126,"type":"VEC2","count":19876,"min":[0.000012357141713437159,0.000010742493941506837],"max":[0.9998306632041931,0.9999417066574097]}],"bufferViews":[{"buffer":0,"byteOffset":0,"byteLength":85358},{"buffer":0,"byteOffset":85360,"byteLength":436343}],"buffers":[{"name":"buffer","byteLength":521704}],"meshes":[{"primitives":[{"attributes":{"POSITION":1,"TEXCOORD_0":2},"indices":0,"material":0,"mode":4,"extensions":{"KHR_draco_mesh_compression":{"bufferView":0,"attributes":{"POSITION":0,"TEXCOORD_0":1}}}}]}],"scene":0,"scenes":[{"nodes":[0]}],"materials":[{"pbrMetallicRoughness":{"baseColorTexture":{"index":0,"texCoord":0},"baseColorFactor":[1,1,1,1],"metallicFactor":1,"roughnessFactor":1},"extensions":{"KHR_materials_unlit":{}},"emissiveFactor":[0,0,0],"alphaMode":"OPAQUE","doubleSided":false}],"textures":[{"sampler":0,"source":0}],"images":[{"name":"textureAtlas","bufferView":1,"mimeType":"image/jpeg"}],"samplers":[{"magFilter":9729,"minFilter":9729,"wrapS":33071,"wrapT":33071}],"nodes":[{"mesh":0}],"extensionsUsed":["KHR_materials_unlit","KHR_draco_mesh_compression"],"extensionsRequired":["KHR_materials_unlit","KHR_draco_mesh_compression"]}

Thanks again for the help.

4

It may be worth noting that this “header” (which is actually just the JSON of the glTF that is contained in the B3DM) contains

"min":[-4495.369140625,-10.204215049743652,-371.314453125],
"max":[-2314.611083984375,90.30708312988281,2300.718505859375]}

for the POSITION accessor. And these values are supposed to be meters. So whatever that geometry is: It is ~2.5 kilometers high. (But from the description so far, it’s hard to tell whether or where some scaling factor may have to be added to correct this…)

5

This may give you a better idea of what I’m seeing when I render to a local tangent plane.

The flat(ish) ground is built from DTED using Bing imagery. That’s the expected orientation for the Melbourne dataset.

image
image1271×1013 134 KB

6

That is an interesting find. It doesn’t seem correct for something in that dataset to be so high. I’m also getting HAE values of -1300 which seems wrong.

If I import this dataset to a local tangent plane then the dataset is at a slant. It seems like the whole dataset is rotated. I know @Kevin_Ring confirmed my math, but it feels like I’m not applying the transform correctly.

Screenshot of the flat horizon and the rotated dataset:

image
image986×1013 49.5 KB

7

Now, from the screenshots alone, it looks like this is just one of the usual cases of where the up-axis is messed up. These should be rare nowadays (in the early days of glTF, there wasn’t really a convention, but now, the up-axis is specified).

Is this data from a tileset where the tileset JSON happens to contain some gltfUpAxis entry somewhere?

As a quick shot, without going through the math and matrix entries, you could just try quickly slamming a Z-up-to-Y-up matrix [1, 0, 0, 0, 0, 0, -1, 0, 0, 1, 0, 0, 0, 0, 0, 1] or a Y-up-to-Z-up matrix [1, 0, 0, 0, 0, 0, 1, 0, 0, -1, 0, 0, 0, 0, 0, 1] to your matrix - that might already fix it.

8

Ah yes Marco is correct, and I forgot about this in my original response.

glTF has a convention that the Y-axis is “up”. When we loaded a glTF (or b3dm) embedded in 3D Tiles, we apply an extra transformation so that a normal Y-up model will instead have its up direction aligned with +Z. This is described in the 3D Tiles spec:

So, there is an extra “secret” Y-up to Z-up transformation that is appled to the vertices before transforming them with the tile’s matrix. This is true even if the tileset doesn’t explicitly specify a gltfUpAxis. However, if the tileset does have that property, the “secret” transformation may be different, or it may be removed entirely (i.e., if the gltfUpAxis is already +Z).

9

I think we’re on to something here but I haven’t been able to get it working yet. For this dataset, gltfUpAxis is not specified.

Unreal Engine is left-handed; with positive X forward, positive Y right, and positive Z up.

(for this dataset) glTF is right-handed; with positive X ??, positive Y ?? and positive Z ??. Can you fill in the ?? I’m trying to get the right conversion from glTF to UE’s coordinate system.

Thanks

10

From the glTF specification:

glTF defines +Y as up, +Z as forward, and -X as right; the front of a glTF asset faces +Z.

So it will likely be necessary to do a y-up-to-z-up conversion, including a change of the handedness.

11

@daktor are you able to convert successfully to ECEF at this point at least? If not, I’d start there. Once you have ECEF coordinates, there’s no single way to go from there to Unreal coordinates. In Cesium for Unreal, we use a CesiumGeoreference Actor to make the Unreal world coordinates a local East-South-Up system at a particular point on the globe. But there are other equally valid ways to do it.

12

Just to be clear, when I say “East-South-Up”, I mean +X points East, +Y points South, and +Z points up (in the direction of the WGS84 ellipsoid surface normal).

13

@Kevin_Ring
I’m not sure if I’m getting the right ECEF. I’m able to transform each vertex to ECEF, then convert to long/lat/HAE. The long/lat of each vertex seems correct; right in the ballpark for Melbourne. However, the HAE of each vertex seems off.

We’ve mapped +X to north, +Y to east, and +Z points up (in the direction of the WGS84 ellipsoid surface normal).

This is the logic I’m using that results in the slanted model screenshots from before:

FTransform datasetTransform = FTransform(
	FMatrix(
	FPlane(-0.5699774740294932, -0.5068551282833045, -0.6467020628015874, 0),
	FPlane(-0.8216313954439031, 0.34498402763987435, 0.4537707248096024, 0),
	FPlane(-0.006894136626272664, 0.7899898098116428, -0.6130812109940204, 0),
	FPlane(-4129944.4597646086, 2897853.584712872, -3889026.1577356746, 1)
));

FVector vertECEF = datasetTransform.TransformPosition(rawVertexFromModel);

I attempt to switch from y-up to z-up (and right to left-handedness) by changing the rawVertexFromModel. This (hopefully) converts from glTF (+z forward, +y up, +x left) to Unreal (+x forward, +y right, +z up). This results in the model slanted with less of an angle, but still not correct.

double x = rawVertexFromModel.X;
double y = rawVertexFromModel.Y;
double z = rawVertexFromModel.Z;

rawVertexFromModel.X = z;
rawVertexFromModel.Y = -x
rawVertexFromModel.Z = y;

FVector vertECEF = FVector(datasetTransform.TransformPosition(rawVertexFromModel));

I think our ECEF → ENU code is working, as I can render 3DTile datasets without a transform successfully (e.g. Google 3D Tiles). So far all 3DTile datasets with a transform that I’ve tested have the same issue (slanted), so I think the issue is in how we’re handling the transform from glTF.

14

To transform the glTF coordinate system to the 3D Tiles one, you should use the matrix in the 3D Tiles spec rather than swapping and negating components (it’s possible what you’ve done is equivalent, I haven’t checked, but following the spec will eliminate any doubt).

The matrix (row-major) is:

[
  1.0, 0.0,  0.0, 0.0,
  0.0, 0.0, -1.0, 0.0,
  0.0, 1.0,  0.0, 0.0,
  0.0, 0.0,  0.0, 1.0
]

This should be applied to the glTF vertex positions before applying the tile transform. In other words, you want: ECEF position = Tile Transform * Y-up-to-Z-up (above) * Vertex Position

15

I may be missing something fundamental here.

Unreal is +x forward, +y right, +z up.
glTF is +x left, +y up, +z forward.

So to convert from glTF to Unreal, it seems like we want this transform:

FTransform tileTransform = FTransform(
	FMatrix(
	FPlane(-0.5699774740294932, -0.5068551282833045, -0.6467020628015874, 0),
	FPlane(-0.8216313954439031, 0.34498402763987435, 0.4537707248096024, 0),
	FPlane(-0.006894136626272664, 0.7899898098116428, -0.6130812109940204, 0),
	FPlane(-4129944.4597646086, 2897853.584712872, -3889026.1577356746, 1)
));

FTransform gltfToUnreal = FTransform(
				FMatrix(
				FPlane(0, -1, 0, 0),
				FPlane(0, 0, 1, 0),
				FPlane(1, 0, 0, 0),
				FPlane(0, 0, 0, 1)
));

I’m attempting to get ECEF with the following logic.

FVector vertECEF = tileTransform.TransformPosition(gltfToUnreal .TransformPosition(vertexPosition));

I’ve gone wrong somewhere though, because the model is still slanted.

Additional info (and questions):
It looks like the TileTransform may already handle the y-up to z-up conversion. If I transform the VertexPosition with the TileTransform, then it looks like I get an ECEF with Z up (as shown here: Coordinate Systems). Does the transform node do the y-up to z-up conversion? Or does the transform node give ECEF with y-up?

In other datasets (e.g. google 3d tiles), the ECEF comes in with y-up. (so Y is pointing through top of earth), so I have to swap y and z before calculating LLH from ECEF.

16

Finally got it! It turns out I was not using FTransforms correctly. Thanks for your help.

(all of this is for Unreal devs)

Below is how to get the actual ECEF from a transform specified in 3DTile json. We’ve got world rotation 0 as north, 90 as east, 180 as south, and 270 as west; +Z is up.

“transform”:[-0.5699774740294932,-0.8216313954439031,-0.006894136626272664,0,-0.5068551282833045,0.34498402763987435,0.7899898098116428,0,-0.6467020628015874,0.4537707248096024,-0.6130812109940204,0,-4129944.4597646086,2897853.584712872,-3889026.1577356746,1]

FVector vert = rawVertexPositionFromModel;

FMatrix tileMatrix;
tileMatrix.M[0][0] = -0.5699774740294932; tileMatrix.M[0][1] = -0.5068551282833045; tileMatrix.M[0][2] = -0.6467020628015874; tileMatrix.M[0][3] = -4129944.4597646086;
tileMatrix.M[1][0] = -0.8216313954439031; tileMatrix.M[1][1] = 0.34498402763987435; tileMatrix.M[1][2] = 0.4537707248096024; tileMatrix.M[1][3] = 2897853.584712872;
tileMatrix.M[2][0] = -0.006894136626272664; tileMatrix.M[2][1] = 0.7899898098116428; tileMatrix.M[2][2] = -0.6130812109940204; tileMatrix.M[2][3] = -3889026.1577356746;
tileMatrix.M[3][0] = 0; tileMatrix.M[3][1] = 0; tileMatrix.M[3][2] = 0; tileMatrix.M[3][3] = 1;

FMatrix yToZ;
yToZ.M[0][0] = 1; yToZ.M[0][1] = 0; yToZ.M[0][2] = 0; yToZ.M[0][3] = 0;
yToZ.M[1][0] = 0; yToZ.M[1][1] = 0; yToZ.M[1][2] = 1; yToZ.M[1][3] = 0;
yToZ.M[2][0] = 0; yToZ.M[2][1] = 1; yToZ.M[2][2] = 0; yToZ.M[2][3] = 0;
yToZ.M[3][0] = 0; yToZ.M[3][1] = 0; yToZ.M[3][2] = 0; yToZ.M[3][3] = 1;

FMatrix vertMatrix;
vertMatrix.SetIdentity();
vertMatrix.M[0][0] = vert.X; 
vertMatrix.M[1][0] = vert.Y; 
vertMatrix.M[2][0] = vert.Z; 
vertMatrix.M[3][0] = 1;

FMatrix result = tileMatrix * yToZ * vertMatrix;

FVector vertECEF;
vertECEF.X = result.M[0][0];
vertECEF.Y = result.M[1][0];
vertECEF.Z = result.M[2][0];
1 Like