glTF2.0 in UE5.2, metadata "EXT_structural_metadata", "EXT_mesh_features"

m.kwiatkowski · September 20, 2023, 9:07am

Hi,
I try convert b3dm file to glb file with metadata (“EXT_structural_metadata”,“EXT_mesh_features”). I use last version:

I use convertB3dmToGlb :

npx 3d-tiles-tools convertB3dmToGlb -i ./0.b3dm -o ./0.glb

glb file have all data, when I open in notepad, but UE cant see this metadata.

My question is:

Does metadata work with glTF2.0 files in Cesium UE5.2?

Description My test

download Reichstag.gml from this link https://cesium.com/public/learn/SampleData.zip
convert Reichstag.gml to 0.b3dm file use Cesium server: https://ion.cesium.com/ and download:

image1912×744 145 KB
Put this model to UE and print metadata:

Next I change b3dm to glb use this:

npx 3d-tiles-tools convertB3dmToGlb -i ./0.b3dm -o ./0.glb

Model 0.glb have batch table and correct matrix translate,:

glTF   0Ä  @  JSON{"asset":{"generator":"glTF-Transform","version":"2.0"},"accessors":[{"type":"SCALAR","componentType":5123,"count":11469},{"type":"VEC3","componentType":5126,"count":6091},{"type":"VEC3","componentType":5126,"count":6091,"max":[103.87069702148438,99.94428253173828,51.667388916015625],"min":[-43.64699935913086,-15.401000022888184,-68.08100128173828]},{"type":"SCALAR","componentType":5126,"count":6091},{"type":"SCALAR","componentType":5126,"count":8469,"bufferView":0,"byteOffset":0}],"bufferViews":[{"buffer":0,"byteOffset":0,"byteLength":33876},{"buffer":0,"byteOffset":33876,"byteLength":13122},{"buffer":0,"byteOffset":46998,"byteLength":4},{"buffer":0,"byteOffset":47002,"byteLength":4},{"buffer":0,"byteOffset":47006,"byteLength":4},{"buffer":0,"byteOffset":47010,"byteLength":4},{"buffer":0,"byteOffset":47014,"byteLength":8},{"buffer":0,"byteOffset":47022,"byteLength":12},{"buffer":0,"byteOffset":47034,"byteLength":8},{"buffer":0,"byteOffset":47042,"byteLength":9},{"buffer":0,"byteOffset":47051,"byteLength":8}],"buffers":[{"name":"buffer","byteLength":47059}],"materials":[{"name":"materialBatched_Building_BLDG_1140001MeshColoredPrimitiveGroundSurface","doubleSided":true,"pbrMetallicRoughness":{"roughnessFactor":0,"metallicFactor":0}}],"meshes":[{"name":"Batched_Building_BLDG_1140001Mesh","primitives":[{"attributes":{"NORMAL":1,"POSITION":2,"_FEATURE_ID_0":3},"mode":4,"material":0,"indices":0,"extensions":{"KHR_draco_mesh_compression":{"bufferView":1,"attributes":{"NORMAL":0,"POSITION":1,"_FEATURE_ID_0":2}},"EXT_mesh_features":{"featureIds":[{"featureCount":1,"attribute":0,"propertyTable":0}]}}}]}],"nodes":[{"mesh":0},{"translation":[3783816.4731581276,5038125.0065327035,-899774.5599196975],"children":[0]}],"scenes":[{"name":"defaultScene","nodes":[1]}],"scene":0,"extensionsUsed":["KHR_draco_mesh_compression","EXT_structural_metadata","EXT_mesh_features"],"extensionsRequired":["KHR_draco_mesh_compression"],"extensions":{"EXT_structural_metadata":{"schema":{"id":"ID_batch_table","name":"Generated from batch_table","classes":{"class_batch_table":{"name":"Generated from batch_table","properties":{"Height":{"name":"Height","description":"Generated from Height","type":"SCALAR","componentType":"FLOAT32","required":true},"Latitude":{"name":"Latitude","description":"Generated from Latitude","type":"SCALAR","componentType":"FLOAT32","required":true},"Longitude":{"name":"Longitude","description":"Generated from Longitude","type":"SCALAR","componentType":"FLOAT32","required":true},"bldg_rooftype":{"name":"bldg:rooftype","description":"Generated from bldg:rooftype","type":"STRING","required":true},"gml_id":{"name":"gml:id","description":"Generated from gml:id","type":"STRING","required":true},"gml_name":{"name":"gml:name","description":"Generated from gml:name","type":"STRING","required":true}}}}},"propertyTables":[{"name":"Property Table","class":"class_batch_table","count":1,"properties":{"Height":{"values":2},"Latitude":{"values":3},"Longitude":{"values":4},"bldg_rooftype":{"values":5,"stringOffsets":6},"gml_id":{"values":7,"stringOffsets":8},"gml_name":{"values":9,"stringOffsets":10}}}]}}}

but I cant read this metadata in UE5.

Kevin_Ring · September 20, 2023, 11:53am

We’re in the process of adding support for those newer extensions. It’s in this PR:

github.com/CesiumGS/cesium-unreal

Rework `UCesiumEncodedMetadataComponent` -> `UCesiumFeaturesMetadataComponent`

CesiumGS:upgrade-feature-metadata ← CesiumGS:encoded-metadata-rework

opened 06:59PM - 11 Aug 23 UTC

j9liu

+24118 -4103

Part of the large metadata system overhaul. This PR does the following: - Dep…recates `UCesiumEncodedMetadataComponent`. - Includes backward compatibility, so styling should still work. - Removed "Auto Fill" and "Generate Material" functions, to encourage people to upgrade to the new component. - Adds functions for handling different methods of encoding in `CesiumEncodedMetadataConversions.h` - Deprecates internal namespace `CesiumEncodedMetadataUtility`. Use `CesiumEncodedFeaturesMetadata` instead. - Incorporated ability to parse RGB colors from string from [`metadata-colors`](https://github.com/CesiumGS/cesium-unreal/tree/metadata-colors) (delete that branch after this change?) - Adds support for encoding implicit feature IDs (passes them in a texcoord like other feature ID attributes) - Adds `UCesiumFeaturesMetadataComponent`. - Now displays descriptions of feature ID sets, separate from property tables - Property table properties now display their _actual_ type info via `FCesiumMetadataPropertyDetails`. This way, unsupported properties can show up and users can decide if want to try encoding anyways. - Encoding info is displayed / changed via `FCesiumMetadataEncodingDetails`. - This allows the user to specify how a property should attempt to be encoded, e.g. "Coerce" or "Parse Color From String". - This can be expanded in C++ for custom encoding functionality. - "Generate Material" now generates nodes for feature ID sets without requiring them to be connected to property tables - "Generate Material" detects when trying to change an asset with unsaved changes, to avoid a crash - Updates the upgrade guide for most of the changes Example styling on San Francisco Ferry Building, using "Parse Color from String" as encoding method: ![image](https://github.com/CesiumGS/cesium-unreal/assets/32226860/99405c92-48f8-404a-af35-23c54c1565cf) This PR does NOT address the following: - Property textures - ~Normalized properties (will leave comment)~ - ~Null feature IDs -- will result in undefined property access in materials, e.g. random property colors will show up~ - ~Feature IDs aren't working for point cloud attenuation because the vertex factory isn't passing them in. It seems like the texcoord-related parameters are never binding, but I don't understand why :/~

In the meantime, B3DM and EXT_feature_metadata are currently supported. We recommend you avoid the latter, though, because support for it will be removed when we add support for the newer extensions.

m.kwiatkowski · September 21, 2023, 8:03am

Thx, for sample gltf!
For me work fine. changed texture if element ID is odd

I adapted Your model:
https://github.com/CesiumGS/cesium-unreal/files/12613418/TestStructuralMetadata.zip
To old OSM meatdata atributes. Change name (building to defaul, and Height to cesium#estimatedHeight and id to elementId) and put model to Warsawa:
Here is my gltf

{
  "accessors": [
    {
      "bufferView": 0,
      "byteOffset": 0,
      "componentType": 5126,
      "count": 240,
      "type": "VEC3",
      "min": [
        -103.97583675780334,
        -114.61250572279096,
        -102.50925003085285
      ],
      "max": [
        118.4458890708629,
        106.54341480974108,
        93.78807587129995
      ]
    },
    {
      "bufferView": 1,
      "byteOffset": 0,
      "componentType": 5126,
      "count": 240,
      "type": "VEC3",
      "min": [
        -0.9686356343768793,
        -0.7415555652213446,
        -0.765567091384559
      ],
      "max": [
        0.9686356343768793,
        0.7415555652213446,
        0.765567091384559
      ]
    },
    {
      "bufferView": 2,
      "byteOffset": 0,
      "componentType": 5126,
      "count": 240,
      "type": "SCALAR",
      "min": [
        0
      ],
      "max": [
        9
      ]
    },
    {
      "bufferView": 3,
      "byteOffset": 0,
      "componentType": 5123,
      "count": 360,
      "type": "SCALAR",
      "min": [
        0
      ],
      "max": [
        239
      ]
    }
  ],
  "asset": {
    "generator": "3d-tiles-samples-generator",
    "version": "2.0"
  },
  "buffers": [
    {
      "name": "buffer",
      "byteLength": 7520,
      "uri": "buffer.bin"
    }
  ],
  "bufferViews": [
    {
      "buffer": 0,
      "byteLength": 2880,
      "byteOffset": 0,
      "target": 34962,
      "byteStride": 12
    },
    {
      "buffer": 0,
      "byteLength": 2880,
      "byteOffset": 2880,
      "target": 34962,
      "byteStride": 12
    },
    {
      "buffer": 0,
      "byteLength": 960,
      "byteOffset": 5760,
      "target": 34962,
      "byteStride": 4
    },
    {
      "buffer": 0,
      "byteLength": 720,
      "byteOffset": 6720,
      "target": 34963
    },
    {
      "buffer": 0,
      "byteLength": 40,
      "byteOffset": 7440
    },
    {
      "buffer": 0,
      "byteLength": 40,
      "byteOffset": 7480
    }
  ],
  "materials": [
    {
      "pbrMetallicRoughness": {
        "baseColorFactor": [
          1,
          1,
          1,
          1
        ],
        "roughnessFactor": 1,
        "metallicFactor": 0
      },
      "alphaMode": "OPAQUE",
      "doubleSided": false,
      "emissiveFactor": [
        0,
        0,
        0
      ]
    }
  ],
  "meshes": [
    {
      "primitives": [
        {
          "attributes": {
            "POSITION": 0,
            "NORMAL": 1,
            "_FEATURE_ID_0": 2
          },
          "indices": 3,
          "material": 0,
          "mode": 4,
          "extensions": {
            "EXT_feature_metadata": {
              "featureIdAttributes": [
                {
                  "featureTable": "default",
                  "featureIds": {
                    "attribute": "_FEATURE_ID_0"
                  }
                },
                {
                  "featureTable": "default",
                  "featureIds": {
                    "constant": 0,
                    "divisor": 2
                  }
                }
              ]
            }
          }
        }
      ]
    }
  ],
  "nodes": [
    {
      "matrix": [
        1,
        0,
        0,
        0,
        0,
        0,
        -1,
        0,
        0,
        1,
        0,
        0,
        0,
        0,
        0,
        1
      ],
      "name": "rootNode",
      "children": [
        1
      ]
    },
    {
      "name": "RTC_CENTER",
      "mesh": 0,
      "translation": [
        3657128.468613,
        1395388.641591,
        5019168.617769
      ],
      "rotation": [
        0,
        0,
        0,
        1
      ],
      "scale": [
        1,
        1,
        1
      ]
    }
  ],
  "scene": 0,
  "scenes": [
    {
      "nodes": [
        0
      ]
    }
  ],
  "extensionsUsed": [
    "EXT_feature_metadata"
  ],
  "extensionsRequired": [
    "EXT_feature_metadata"
  ],
  "extensions": {
    "EXT_feature_metadata": {
      "schema": {
        "classes": {
          "default": {
            "properties": {
              "cesium#estimatedHeight": {
                "type": "FLOAT32",
				"default": 0.0
              },
              "elementId": {
                "type": "INT32"
              },
              "year": {
                "type": "STRING",
                "optional": true,
                "default": "2022"
              }
            }
          }
        }
      },
      "featureTables": {
        "default": {
          "class": "default",
          "count": 10,
          "properties": {
            "cesium#estimatedHeight": {
              "bufferView": 4
            },
            "elementId": {
              "bufferView": 5
            }
          }
        }
      }
    }
  }
}

But now I need to Know how change buffer.bin.

m.kwiatkowski · September 22, 2023, 6:12am

I already know how to change existing metadata. Fot example elementId have value 0-9 and start in 7480:

But I don’t know how to inject my own metadata. My question is:

Is there any program to create models with metadata?
is there any documentation on how .bin files are created?

Kevin_Ring · September 22, 2023, 6:39am

Every bit of 3D Tiles is covered by detailed and open specifications. The top-level spec documentation is here:

Specifics of glTF as a 3D Tiles tile format are here:

I believe the .bin file you’re looking at in this case is a glTF Buffer, and its structure is defined by the BufferViews and Accessors in the glTF file. If you’re not familiar with these concepts, the glTF spec is the place to start:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html

m.kwiatkowski · September 22, 2023, 8:01am

THX, now I have all what I need.
I separate model to ( buffer.bin and bufferMeta.bin):

  "buffers": [
    {
      "name": "buffer",
      "byteLength": 7520,
      "uri": "buffer.bin"
    },
	{
      "name": "bufferMeta",
      "byteLength": 81,
      "uri": "bufferMeta.bin"
    }
  ],

now all works fine! Thx.

Marco13 · September 22, 2023, 1:55pm

There is no specific program for adding metadata to tiles. Some of the required functionality for that does already exist in the 3d-tiles-tools. This is not exposed as an official/public functionality - so the implementation may change arbitrarily, at any point in time. But a very basic demo of the relevant classes that can be used to create glTF assets with the EXT_structural_metadata extension can be found in the ExtStructuralMetadataPropertyTable demo. If you have any feedback or ideas about how that should be made available, feel free to drip your ideas here or in an issue in the 3d-tiles-tools repo.

m.kwiatkowski · September 26, 2023, 6:07am

@Marco13 Thx. But now I know How injected metadata to own buildings:
here is my example:

budynki.zip (217.5 KB)

I create buildings in Blender with _FEATURE_ID_0:

image912×660 78.7 KB
Connect vertexes to _FEATURE_ID_0

image960×509 84.3 KB
Export gltTF with attribute table:
Create own binary file with metadata:

image963×110 7.41 KB

Form me cesium#estimatedHeight :

FLOAT32 big-endian hexa 0x00002041 = 10
FLOAT32 big-endian hexa 0x0000F041 = 30
FLOAT32 big-endian hexa 0x00004842 = 50

elemetID:

INT32 hexa 0x00000000 = 0
INT32 hexa 0x01000000 = 1
INT32 hexa 0x02000000 = 2

Connect metadata file to glt:

    "buffers":[
        {
            "byteLength":3384,
            "uri":"budynki.bin"
        },
        {
          "name": "bufferMeta",
          "byteLength": 24,
          "uri": "bufferMeta.bin"
        }
    ],

Create bufferViews with metadata:

    "bufferViews":[
        {
          ...
        },
        {
          "buffer": 1,
          "byteLength": 12,
          "byteOffset": 0
        },
        {
          "buffer": 1,
          "byteLength": 12,
          "byteOffset": 12
        }
    ],

add accessors with metadata:

    "accessors":[
        {
          ..
        },
        {
            "bufferView":6,
            "componentType":5126,
            "count":12,
            "type":"SCALAR",
            "max":[
                50
            ],
            "min":[
                10
            ]
        },
        {
            "bufferView":7,
            "componentType":5125,
            "count":12,
            "type":"SCALAR",
            "max":[
                2
            ],
            "min":[
                0
            ]
        }
    ],

connect to mesh:

    "meshes":[
        {
            "name":"Cube.003",
            "primitives":[
                {
                    "attributes":{
                        "_FEATURE_ID_0":0,
                        "COLOR_0":1,
                        "POSITION":2,
                        "NORMAL":3,
                        "TEXCOORD_0":4
                    },
                    "indices":5,
                    "material":0,
                    "extensions": {
                      "EXT_feature_metadata": {
                        "featureIdAttributes": [
                          {
                            "featureTable": "default",
                            "featureIds": {
                              "attribute": "_FEATURE_ID_0"
                            }
                          },
                          {
                            "featureTable": "default",
                            "featureIds": {
                              "constant": 0,
                              "divisor": 2
                            }
                          }
                        ]
                      }
                    }
                }
            ]
        }
    ],

Add class:

      "extensionsUsed": [
        "EXT_feature_metadata"
      ],
      "extensionsRequired": [
        "EXT_feature_metadata"
      ],
      "extensions": {
        "EXT_feature_metadata": {
          "schema": {
            "classes": {
              "default": {
                "properties": {
                  "cesium#estimatedHeight": {
                    "type": "FLOAT32",
                    "default": 0.0
                  },
                  "elementId": {
                    "type": "INT32"
                  },
                  "year": {
                    "type": "STRING",
                    "optional": true,
                    "default": "2022"
                  }
                }
              }
            }
          },
          "featureTables": {
            "default": {
              "class": "default",
              "count": 3,
              "properties": {
                "cesium#estimatedHeight": {
                  "bufferView": 6
                },
                "elementId": {
                  "bufferView": 7
                }
              }
            }
          }
        }
      }

All works finie in UE.

This example change tekstur when heigh is 30:

But now I need write python script to Blender and automate this process. And I need to know how to use EXT_mesh_gpu_instancing

Marco13 · September 30, 2023, 1:07pm

Thank you @m.kwiatkowski for this detailed description!

It seems like there are some manual steps in this process (so you probably have to edit JSON files with a text editor, eventually - is that correct?)

But in general, a description of how to assign custom attributes (like the _FEATURE_ID_0) in Blender could be really useful. I’ll try to allocate some time to try out the process that you described, and see where and how users could be supported in that process, either by providing additional tools or scripts, or by providing more details about the (manual) modifications.

We’ll consider to either link to this post when the question comes up in the future. Maybe we can even create some learning material with a description of this process and/or additional details.

m.kwiatkowski · October 2, 2023, 7:48am

@Marco13

If you want, I can record a video and show you step by step how to do it.
Now trying to find a working example in UE with EXT_mesh_gpu_instancing.

Marco13 · October 2, 2023, 2:09pm

If you want, I can record a video and show you step by step how to do it.

As I said, I’ll try to find some time to go through the process, but it’s hard to give a time estimate. Very broadly speaking: Creating and editing metadata is a topic that also comes up elsewhere, and there are many ways in which this could be addressed.

Now trying to find a working example in UE with EXT_mesh_gpu_instancing.

I had created a SimpleInstancing example a while ago, which can be found at https://github.com/KhronosGroup/glTF-Sample-Assets/tree/main/Models/SimpleInstancing .

Also, the GpuInstancesMetadata at https://github.com/CesiumGS/3d-tiles-samples/tree/main/glTF/GpuInstancesMetadata that combines all the extensions that are involved here.

janine · October 2, 2023, 3:05pm

Now trying to find a working example in UE with EXT_mesh_gpu_instancing.

Just wanted to give a headsup that EXT_mesh_gpu_instancing is not currently supported in Cesium for Unreal. There was a Github PR to add support, but it has not had any updates in years, and we don’t have a time estimate for when we will properly address it. Apologies for the inconvenience.

IndigoYun · October 7, 2023, 3:23am

Does Cesium for unreal support read 3DTiles1.1 metadata ?

Kevin_Ring · October 8, 2023, 10:38am

The recently-released v2.0 preview does.
https://community.cesium.com/t/cesium-for-unreal-v2-0-preview-release/27181/2

m.kwiatkowski · October 11, 2023, 11:34am

Hi,

I checked cesium 2.0.
I adapted my previous design to the new standard.

Here is project (gltf with EXT_mesh_features + tileset.json + blender file ):
budynki.zip (217.7 KB)
But in new Cesium texture size is too small. I need scale all texture by 10000.

only then the textures look normal

But great work!

Topic		Replies	Views
Metadata GLTF2.0 in Cesium 2.7.0 Cesium for Unreal	6	51	August 2, 2024
Including metadata in gltf Cesium for Unreal	5	671	June 14, 2023
Metadata sample with EXT_Mesh_Features/EXT_Structural_Metadata? Cesium for Unity	4	233	December 12, 2023
Gltf feature extensions support in Cesium, EXT_mesh_features 3D Tiles	6	526	January 12, 2024
Add 3dtiles(extension) Cesium for Unreal	5	1025	October 27, 2021

glTF2.0 in UE5.2, metadata "EXT_structural_metadata", "EXT_mesh_features"

Related topics