Gltf embed metadata, EXT_mesh_features, EXT_structural_metadata

3D Tiles
1

I am working on the custom tileset.json, and I have the valid gltf files now, I want to embed the metadata to the corresponding gltf files.

Description: I am working on the powerlines network, there are many of the powerlines more than 10,000 of them. A powerline is a curved line. I want to embed the metadata to each of the gltf, so I could use the 3D style condition to dynamically apply style to it like this:

const voltageStyle = new Cesium.Cesium3DTileStyle({
    color: {
        conditions: [
            ["${Voltage} === 240", "color('yellow')"], // Show tiles where Voltage is 240 or higher
            ["${Voltage} > 0", "color('red')"], // Show tiles where Voltage is 240 or higher
            ["true", "color('blue')"], // Hide all other tiles
        ],
    },
});

and here is the sample sandcastle example to describe the powerline loaded as tileset, I am still updating my latest one on sandcastle.

Task: The expected feature for now is:

  1. style with different color based on voltage
  2. mouse event to show info box for its corresponding data for each single powerline

Method tried: I have tried the tileset metadata method MetadataGranularities, but a grouped tileset can only apply one style to that tileset, I mean it couldn’t change a single part style inside that tileset. I successfully used this method to show corresponding metadata to the parts of the grouped tileset.

Difficulties/issue:
I had multiple tries with the gltf but still cannot figure out how to use extensions, and I followed EXT_structural_metadata/FeatureIdAttributeAndPropertyTable this when I tried. Here is one of my gltf file, it is a powerline like a curved line made of 11 cartesian points, the coordinate system is using Cesium when converting. When I created those powerline, it is grouped based on its tiles coordinate.

To be more specific, I think my issue would be assigning feature id and assigning feature id to property table.

{
    "asset": {
        "version": "2.0"
    },
    "scenes": [
        {
            "nodes": [0]
        }
    ],
    "nodes": [
        {
            "matrix": [1, 0, 0, 0, 0, 0, -1, 0, 0, 1, 0, 0, 0, 0, 0, 1],
            "children": [1]
        },
        {
            "mesh": 0
        }
    ],
    "meshes": [
        {
            "primitives": [
                {
                    "mode": 3,
                    "attributes": {
                        "POSITION": 0
                    }
                }
            ]
        }
    ],
    "buffers": [
        {
            "uri": "data:application/octet-stream;base64,JLKZwnlUUkI2QO3C0EebwuMlU0IR4erC6PmcwskbVEJIoejCPciewu81VUKlgObCvbKgwj50VkIVf+TCcrmiwsLWV0KinOLCg9ykwq1dWUJ42eDCMRynwlQJW0LgNd/C33ipwjPaXEJEst3CCPOrwuzQXkIuT9zCR4uuwkTuYEJKDdvC",
            "byteLength": 132
        }
    ],
    "bufferViews": [
        {
            "buffer": 0,
            "byteOffset": 0,
            "byteLength": 132,
            "target": 34962
        }
    ],
    "accessors": [
        {
            "bufferView": 0,
            "byteOffset": 0,
            "componentType": 5126,
            "count": 11,
            "type": "VEC3",
            "max": [-76.8479318455793, 56.23268031002954, -109.52595210261643],
            "min": [-87.27202824316919, 52.58249282790348, -118.62541104014963]
        }
    ]
}

here is the data to this gltf

[
    {
        "Bay_Id": "173840",
        "ConductorId": "514490",
        "Conductor_Length": "14.537233",
        "Coordinates": [
            {
                "longitude": 2.570975183209337,
                "latitude": -0.7482747440108009,
                "height": 148.9212520894381
            },
            {
                "longitude": 2.570975237863152,
                "latitude": -0.7482745243966467,
                "height": 148.68407230428997
            },
            {
                "longitude": 2.570975292516946,
                "latitude": -0.7482743047824905,
                "height": 148.53690032648962
            },
            {
                "longitude": 2.5709753471707173,
                "latitude": -0.7482740851683327,
                "height": 148.47915607742527
            },
            {
                "longitude": 2.570975401824467,
                "latitude": -0.7482738655541729,
                "height": 148.51061195803672
            },
            {
                "longitude": 2.570975456478194,
                "latitude": -0.748273645940011,
                "height": 148.63139195173324
            },
            {
                "longitude": 2.5709755111318997,
                "latitude": -0.7482734263258474,
                "height": 148.84197211307438
            },
            {
                "longitude": 2.5709755657855835,
                "latitude": -0.7482732067116817,
                "height": 149.14318244413997
            },
            {
                "longitude": 2.5709756204392455,
                "latitude": -0.7482729870975143,
                "height": 149.53621016598464
            },
            {
                "longitude": 2.5709756750928854,
                "latitude": -0.7482727674833449,
                "height": 150.02260439807182
            },
            {
                "longitude": 2.570975729746503,
                "latitude": -0.7482725478691736,
                "height": 150.6042822641308
            }
        ],
        "Voltage": "1245.0",
        "Tile": { "x": 119168, "y": 48377, "level": 16 }
    }
]

and here is the tileset.json that I loaded to the scene

{
  "asset": {
    "version": "1.1"
  },
  "geometricError": 4096,
  "root": {
    "boundingVolume": {
      "box": [
        -82.05997848510742,
        114.07568359375,
        54.4075870513916,
        5.212047576904297,
        0,
        0,
        0,
        -4.5497283935546875,
        0,
        0,
        0,
        1.825094223022461
      ]
    },
    "geometricError": 512,
    "content": {
      "uri": "514490.glb"
    },
    "refine": "ADD"
  }
}

please leave any ideas to my question or leave a correct gltf if it is possible, many thanks. If need more info regarding to this, please leave any comments as well.

2

There are many degrees of freedom for structuring data sets in general. And the point of metadata adds a whole new dimension to these existing degrees of freedom. The question about what is “The Right Structure” is an engineering question. It is impossible to give profound recommendations here, without a diligent and detailed analysis of the requirements and goals.

For each possible solution, there are pros and cons, or possible caveats and constraints. As mentioned in an earlier thread, where you asked about this, one constraint is that there currently appears to be a bug in CesiumJS where styling based on metadata is not applied.

Therefore, the following should be considered as a “workaround”. It is not ‘the recommended solution’. It only shows one way of how it might be possible to achieve what you want, involving many, many guesses and assumptions.

I took the glTF file that you provided, and created two additional copies of that (moved about 10 and 20 in x-direction, just to have them at different positions). I created a tileset.json that refers to these glTF files. (And this uses a proper bounding box - the one that you provided did not match the data).

In that tileset, I defined a metadata schema - roughly corresponding to the metadata that you provided. (I only omitted the ‘tile’ information. You’ll have to think about how to encode that sensibly. The structure of "Tile": { "x": 119168, "y": 48377, "level": 16 } does not directly match one of the metadata types, but you could store it as a VEC3/INT32 or so - similar to how I stored the coordinates as a VEC3/FLOAT32 for this example)

The tileset refers to each of the GLB files. Each GLB file is one ‘tile’ in the tileset. Each of these tiles has metadata that matches the schema.

The only difference in these tile metadata entities is the Voltage property: It’s 123, 1234, and 12345 for the different tiles.

The result is in this archive:

Forum-29582-example.zip (5.3 KB)

The archive contains a Sandcastle.js that shows how this metadata might be used. The sandcastle inspects the tiles that are loaded, looks at the metadata of these tiles, and applies different styles to the content of these tiles, depending on the Voltage value:

  • For Voltage < 1000, the content is green
  • For Voltage > 1000, the content is yellow
  • For Voltage > 10000, the content is red

Again: This is a workaround for the styling bug linked above. When you say that you have 10000 tiles/powerlines, then this may cause some performance issues. It might be possible to find a better solution here, but it is intended as one example/demo.

The following is a short screencapture of that sandcastle: The power lines are rendered with different colors. The tooltip shows why - namely, because they have different Voltage values.

Cesium Forum 29582

Once more: This is not “THE” recommended solution. Whether or not this suits your needs is hard to tell. But it shows one possible approach for assigning metadata to tiles, and using that metadata for styling and inspection.

3

Thanks for the reply, I thought it is a different topic using the gltf extensions, that’s why I create another one. Yes I am trying your suggested approach, and I am processing the gltf to tileset.json

But again thank you so much for the help, without your help, I was stuck at this stage in the last two month, and this workaround seems to be ok, I will keep trying this approach and also the gltf extension, and will update you if got progress.

4

As I said in the previous thread (and in the first paragraph here): The question of what the metadata is associated with largely depends on the intended granularity.

When you have one glTF asset that represents “the powerline”, and that is directly used as the tile content, then you can associate the metadata with the tile directly, and don’t need the glTF metadata extensions.

However, when you have a complex glTF that contains multiple components, and when you even want to associate different metadata entities with individual vertices or texels within the glTF data, then you can use the glTF metadata extensions.

5

Hi, here is my work following your suggestions. data embed to tileset example, thanks for the help.

It is working good, and to use more of the gltf features I got a new task which is to use gltf to embed the data. So I am back to the issue with EXT_structure_metadata and EXT_mesh_features.

here is the example file I am working on. A gltf contains three conductors, which are generated from the data file. json file for the tileset, and a glb file.
The gltf’s buffer part uri is for the three conductors, I haven’t put the metadata to buffer yet, because I don’t know what structure it is for the data.
119227.zip (4.7 KB)

My question is:

  1. How to put metadata to the gltf buffer and use properties value point to it. (I already have the schema, and I assume the properties value points to bufferView, and data stored at buffer)
  2. How the featureId points to the property table, how does cesium know which values assign to which mesh.(my gltf have distinct mesh, each mesh primitives represent a conductor)

I have been browsing the doc regarding to this, here is the reference:

and here is how I generate for the buffer for a single conductor using its cartesian coords

export function floatsToBase64(floats) {
    // 1. Place the floats into a Float32Array
    const floatArray = new Float32Array(floats);

    // 2. Convert the buffer of the Float32Array into a Uint8Array
    const uint8Array = new Uint8Array(floatArray.buffer);

    // 3. Convert the Uint8Array to a Base64 string
    let base64 = '';
    const chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';

    for (let i = 0; i < uint8Array.length; i += 3) {
        const a = uint8Array[i];
        const b = uint8Array[i + 1];
        const c = uint8Array[i + 2];
        const index1 = a >> 2;
        const index2 = ((a & 3) << 4) | (b >> 4);
        const index3 = isNaN(b) ? 64 : ((b & 15) << 2) | (c >> 6);
        const index4 = isNaN(b) || isNaN(c) ? 64 : c & 63;

        base64 += chars[index1] + chars[index2] + chars[index3] + chars[index4];
    }

    // Handle padding if necessary
    const paddingLength = 4 - (base64.length % 4);
    if (paddingLength !== 4) {
        base64 += '='.repeat(paddingLength);
    }

    return base64;
}

Any advices are welcome and appreciated. Thanks

6

I’m not exactly sure what the intention is behind this buffer creation code. The question of how you can bring the data for these extensions into the glTF file mainly depends on how you are generating the glTF file in the first place. You will need a glTF reader/writer that supports these extensions (you certainly don’t want to assemble the binary data and JSON manually, particularly when the binary data contains arrays).

There is an implementation of these extensions, based on glTF-Transform, in the 3d-tiles-tools. This is not really “public API”, but there is a demo/test for this functionality in ExtStructuralMetadataPropertyTableDemo.ts (which is part of this state at the time of writing this. Right now, this is the same as the main state, but might change).

It creates a glTF asset with EXT_structural_metadata. There is a similar demo for EXT_mesh_features in the same directory. With this functionality, it should be possible to create glTF assets that contain both. And how this data can be displayed in CesiumJS is shown in the 3D Tiles Samples, specifically, in the Sandcastle that is linked from there.

7

Thanks for the help, just update my progress. I have worked out how to use both extensions.