Hi,
I’m using the Cesium for Unity Samples and would like to visualize a 3D-Model from Cesium Ion.
In one case it works fine:
in the second case I have a model , which isn’t displayed:
Using the same model in Cesium sandcastle both are displayed - one question
Can I display the glb-Model with Cesium for Unity
Regards
Rüdiger
A first shot in such a case is usually to drag-and-drop the GLB files into the glTF Validator and see whether it reports any errors. (CesiumJS is usually a bit more forgiving for invalid/legacy GLB files, so this might give a hint. If the files are valid, then further investigations will be necessary)
Hi Marco,
ok - I check the files.
I’m a little bit suprised, I use the tools
GitHub - CesiumGS/3d-tiles-tools
for the generation of the b3dm from the glb. Now I get a long list, that every glb-File has an Error.
For Examples - i think there is an issue with value null and integer
{
"type": "CONTENT_VALIDATION_INFO",
"path": "/extensionsUsed/0",
"message": "Cannot validate an extension as it is not supported by the validator: 'KHR_spz_gaussian_splats_compression'.",
"severity": "INFO"
},
{
"type": "CONTENT_VALIDATION_ERROR",
"path": "/accessors/0/bufferView",
"message": "Type mismatch. Property value null is not a 'integer'.",
"severity": "ERROR"
},
{
"type": "CONTENT_VALIDATION_ERROR",
"path": "/accessors/0/byteOffset",
"message": "Dependency failed. 'bufferView' must be defined.",
"severity": "ERROR"
},
From the gltf validator I get similar Errors:
"messages": [
{
"code": "UNSUPPORTED_EXTENSION",
"message": "Cannot validate an extension as it is not supported by the validator: 'KHR_spz_gaussian_splats_compression'.",
"severity": 2,
"pointer": "/extensionsUsed/0"
},
{
"code": "TYPE_MISMATCH",
"message": "Type mismatch. Property value null is not a 'integer'.",
"severity": 0,
"pointer": "/accessors/0/bufferView"
},
{
"code": "UNSATISFIED_DEPENDENCY",
"message": "Dependency failed. 'bufferView' must be defined.",
"severity": 0,
"pointer": "/accessors/0/byteOffset"
},
{
"code": "TYPE_MISMATCH",
"message": "Type mismatch. Property value null is not a 'integer'.",
"severity": 0,
"pointer": "/accessors/1/bufferView"
},
{
"code": "UNSATISFIED_DEPENDENCY",
"message": "Dependency failed. 'bufferView' must be defined.",
"severity": 0,
"pointer": "/accessors/1/byteOffset"
},
{
"code": "TYPE_MISMATCH",
"message": "Type mismatch. Property value null is not a 'integer'.",
"severity": 0,
"pointer": "/accessors/2/bufferView"
},
{
"code": "UNSATISFIED_DEPENDENCY",
"message": "Dependency failed. 'bufferView' must be defined.",
"severity": 0,
"pointer": "/accessors/2/byteOffset"
},
{
"code": "TYPE_MISMATCH",
"message": "Type mismatch. Property value null is not a 'integer'.",
"severity": 0,
"pointer": "/accessors/3/bufferView"
},
{
"code": "UNSATISFIED_DEPENDENCY",
"message": "Dependency failed. 'bufferView' must be defined.",
"severity": 0,
"pointer": "/accessors/3/byteOffset"
},
{
"code": "NODE_MATRIX_DEFAULT",
"message": "Do not specify default transform matrix.",
"severity": 2,
"pointer": "/nodes/0/matrix"
},
{
"code": "MESH_PRIMITIVE_ATTRIBUTES_ACCESSOR_INVALID_FORMAT",
"message": "Invalid accessor format '{VEC4, UNSIGNED_BYTE}' for this attribute semantic. Must be one of ('{VEC3, FLOAT}', '{VEC3, UNSIGNED_BYTE normalized}', '{VEC3, UNSIGNED_SHORT normalized}', ..., '{VEC4, UNSIGNED_BYTE normalized}', '{VEC4, UNSIGNED_SHORT normalized}').",
"severity": 0,
"pointer": "/meshes/0/primitives/0/attributes/COLOR_0"
},
{
"code": "UNUSED_OBJECT",
"message": "This object may be unused.",
"severity": 2,
"pointer": "/bufferViews/0"
}
],
I’m not an expert, but this happened in a tooling chain, where I would like to display Gaussian splats in Cesium -
Regards
Rüdiger
Hi Marco,
I would like to ask:
is it possible to visualize a model with Gaussian Splats using Cesium for Unity ?
b3dm: AssetId 3941072
GLB: AssetId 3939354
I think the b3dm-Files are corrupted or not really convert in the right way via the 3d-tiles-tools.
Rüdiger
Some context might be missing here. But a few high-level points:
3d-tiles-tools do not handle Gaussian Splat data at all.KHR_spz_gaussian_splats_compression in one of the error messages that you posted, but the most recent name is KHR_gaussian_splatting_compression_spz_2 (see KHR_gaussian_splatting_compression_spz_2 by weegeekps · Pull Request #2531 · KhronosGroup/glTF · GitHub )So in view of the validation errors, the question is: How exactly did you create the GLB files?
In view of the usage, the question is: Do you need this in Cesium For Unity, or could CesiumJS be sufficient for previewing the Gaussian Splat rendering?
In any case: When you upload Gaussian Splat data in form of PLY files to Cesium ion, then it should convert them into Gaussian Splat GLB files that can be rendered in CesiumJS.
Hi Marco,
I use GisBox GISBox: Free Self-Hosted 3D Tiles Streaming and OSGB to 3D Tiles for the conversion of Gaussian splats to 3d-tiles (GLB).
Then I converted the glb-files with 3d-tiles-tools to B3dm
I tried to display the GLB model in Cesium for Unity, but the Model isn’t displayed.
I thought, that the model isn’t displayed, because it is glb, then I tried to convert to b3dm, because I have a model in b3dm, which is displayed in Cesium for Unity
I tried to upload the ply-File but I got an Error !!
Something went wrong with this asset. To report this issue, please email support@cesium.com with the asset ID: 3944893.
To help resolve this faster, include any relevant information about your data, such as what software you used to export it.
Regards
Rüdiger
The GLBs must not be converted to B3DM. The GLBs must be used directly.
@cesium-ion Maybe someone can have a look at what went wrong with asset 3944893
In the meantime, @Ruediger_Brand , if you can share the header of the PLY file, that might already help. (Just opening it with a text editor should be OK - even if it is a binary file, the header will be ASCII, and it should be possible to copy-and-paste the header here)
Not at the moment. It’s something we would like to implement eventually, but we are currently focused on the implementation in Cesium for Unreal. It’s likely we won’t start on a Cesium for Unity implementation until we finish up with the Unreal implementation.
Hi Marco,
so ok - it is possible to view the GLBs directly - but I don’t see anything, if I use the “add to level” in the Unity - see example Adding Datasets – Cesium
I choose the Asset from Cesium Ion and if click in unity editor, I don’t see it.
→ so this is maybe caused by the wrong splatting Field or doesn’t it work like @azrogers said ?
The header of the ply is:
ply
format binary_little_endian 1.0
comment Generated by SuperSplat 2.8.5
element chunk 18060
property float min_x
property float min_y
property float min_z
property float max_x
property float max_y
property float max_z
property float min_scale_x
property float min_scale_y
property float min_scale_z
property float max_scale_x
property float max_scale_y
property float max_scale_z
property float min_r
property float min_g
property float min_b
property float max_r
property float max_g
property float max_b
element vertex 4623131
property uint packed_position
property uint packed_rotation
property uint packed_scale
property uint packed_color
element sh 4623131
property uchar f_rest_0
property uchar f_rest_1
property uchar f_rest_2
property uchar f_rest_3
property uchar f_rest_4
property uchar f_rest_5
property uchar f_rest_6
property uchar f_rest_7
property uchar f_rest_8
property uchar f_rest_9
property uchar f_rest_10
property uchar f_rest_11
property uchar f_rest_12
property uchar f_rest_13
property uchar f_rest_14
property uchar f_rest_15
property uchar f_rest_16
property uchar f_rest_17
property uchar f_rest_18
property uchar f_rest_19
property uchar f_rest_20
property uchar f_rest_21
property uchar f_rest_22
property uchar f_rest_23
property uchar f_rest_24
property uchar f_rest_25
property uchar f_rest_26
property uchar f_rest_27
property uchar f_rest_28
property uchar f_rest_29
property uchar f_rest_30
property uchar f_rest_31
property uchar f_rest_32
property uchar f_rest_33
property uchar f_rest_34
property uchar f_rest_35
property uchar f_rest_36
property uchar f_rest_37
property uchar f_rest_38
property uchar f_rest_39
property uchar f_rest_40
property uchar f_rest_41
property uchar f_rest_42
property uchar f_rest_43
property uchar f_rest_44
end_header
First of all: As Ashley said, the result can currently not yet be displayed in Unity in any case.
Even when the tiling works.
And… I may have to double-check a few things, but this header does not look like the ~“standard splat PLY header”. The usual splat PLY headers contain
element vertex 1000
property float x
property float y
property float z
property float f_dc_0
property float f_dc_1
property float f_dc_2
property float f_rest_0
property float f_rest_1
property float f_rest_2
property float f_rest_3
property float f_rest_4
property float f_rest_5
property float f_rest_6
property float f_rest_7
property float f_rest_8
property float opacity
property float scale_0
property float scale_1
property float scale_2
property float rot_0
property float rot_1
property float rot_2
property float rot_3
Now it could, in theory, be that this header should also work with ion, and something else is wrong. But the header looks like some proprietary, compressed representation of splats. Are there any export options when writing these PLY files? (Maybe the option to select some ~“Supersplat format” and some ~“Standard format” or so…?)
Hi Marco,
sorry - I’m not an expert and the data is delivered by a customer from us.
It was stated:
These are standard products. The Generator supersplat as well is a standard and also, you stated in another case, the data is displayed well in Babylon.js Sandbox - View glTF, glb, obj and babylon files
If I use SuperSplat for display I can export without compression - I will try this.
It looks like
ply
format binary_little_endian 1.0
element vertex 4623131
property float x
property float y
property float z
property float f_dc_0
property float f_dc_1
property float f_dc_2
property float f_rest_0
property float f_rest_1
property float f_rest_2
property float f_rest_3
property float f_rest_4
property float f_rest_5
property float f_rest_6
property float f_rest_7
property float f_rest_8
property float f_rest_9
property float f_rest_10
property float f_rest_11
property float f_rest_12
property float f_rest_13
property float f_rest_14
property float f_rest_15
property float f_rest_16
property float f_rest_17
property float f_rest_18
property float f_rest_19
property float f_rest_20
property float f_rest_21
property float f_rest_22
property float f_rest_23
property float f_rest_24
property float f_rest_25
property float f_rest_26
property float f_rest_27
property float f_rest_28
property float f_rest_29
property float f_rest_30
property float f_rest_31
property float f_rest_32
property float f_rest_33
property float f_rest_34
property float f_rest_35
property float f_rest_36
property float f_rest_37
property float f_rest_38
property float f_rest_39
property float f_rest_40
property float f_rest_41
property float f_rest_42
property float f_rest_43
property float f_rest_44
property float opacity
property float scale_0
property float scale_1
property float scale_2
property float rot_0
property float rot_1
property float rot_2
property float rot_3
end_header
Rüdiger
Hi Marco,
with the uncompressed data the import to ion it works - but I need 5-times storage !
But the model couldn’t be displayed in Cesium for Unity if I choose “Splatting” ? Is this right ?
Rüdiger
Yes, that header looks like what I’d expect.
An aside: There’s currently a ton of research going on about splats. This refers to the acquisition of the data and generating the splats, and details of the rendering process, but also to the way how splats are stored. There often are certain compression methods that someone published in some reseach paper. There may be some python-based GitHub project with some hard-wired paths and 12 gigabytes of dependencies that only work on a certain OS with a certain graphics card driver. It may be that the SuperSplat format is more widely adopted. But the fact that splats are usually distributed as PLY files is just admitting that: “We have a bunch of un(der)specified (float) values, and don’t know what’s the best way to store them - let’s use a file format that lets us store a bunch of float values”. The property names and value ranges are not specified. Not even the coordinate systems are specified. Things like KHR_gaussian_splatting by weegeekps · Pull Request #2490 · KhronosGroup/glTF · GitHub are efforts to bring some standardization into that. But as you can see (by 193 comments at the time of writing this): It’s complicated. If you have strong opinions about color gamuts, ICC and color encodings, sRGB and BT.709: Join the party 
The fact that the PLY file is 5 times larger is not surprising. It’s uncompressed, and just … a bunch of float values. After ion processed it, the data should be compressed, and much smaller.
That said: Cesium for Unity currently cannot render splats.