Create KHR_spz_gaussian_splats_compression tooling?

3D Tiles
1

Hi, was checking the KHR_spz_gaussian_splats_compression glTF extension (glTF/extensions/2.0/Khronos/KHR_spz_gaussian_splats_compression at draft-spz-splat-compression · CesiumGS/glTF · GitHub), Q: is there some tooling available to create these glTF’s (from spz)?

1 Like
2

@Jason_Sobotka is supposed to talk about the extension on 2025-06-04T17:00:00Z see: Gaussian Splats Town Hall Part 3: Research Insights and Interoperability - Metaverse Standards Forum

1 Like
3

There currently is not (yet) much tooling support for this. The extension itself as well as the support in CesiumJS are currently under heavy development. Depending on the progress and feedback, people can expect to see better support here soon.

However, the extension in its current form is not very complicated. It mainly defines a set of special mesh primitive attributes. (The names of these attributes may have to revised, in view of Disambiguation of attribute semantic names · Issue #2111 · KhronosGroup/glTF · GitHub , but that’s a detail for now). These attributes are given as accessors without buffer views. And they are basically just “filled” with the data that is taken from a plain SPZ file (which is just stored in the buffer view that is set in the extension object itself).

So it’s not even strictly necessary to fully read or process the SPZ data in order to create such a glTF. It is enough to know the number of points (and the number of spherical harmonics degrees) which are just written in the first few bytes of the file header. Apart from that, the whole SPZ can just be dumped into a buffer view, as it is.

Note that right now (as far as I know) such a glTF can not be loaded as a Model directly in CesiumJS. It has to be part of a tileset. But wrapping a small tileset around such a glTF is also not terribly difficult.

And although I’m pretty sure that this will bite back in one way or another, I’m posting it anyhow:

I just created a Java loader for SPZ, partially based on the original SPZ implementation, and added an SpzToTileset example.

Note: This was a pure spare time task. The project is not related to Cesium!

Given an SPZ file, it writes the glTF and the matching tileset JSON into some output directory. This is currently all hard-wired - I just used one of the examples from niantic (not included in the repo). The example SPZ has ~18MB, and that’s a bit too large for a sensible tile content. It should be broken down into smaller tiles, to form a proper tileset with LOD and whatnot.

However, the result can then be loaded with the branch behind the (DRAFT!) PR that adds SPZ support to CesiumJS:

Cesium Gaussians 0001
Cesium Gaussians 00011216×426 396 KB

Again: Everything is pretty much in flux here. It might be that none of that works … tomorrow. But the main point is: Adding support for this extension on the producing side should be relatively easy. (Most of the work has to be done by the consuming side - there’s always some trade-off for that…)

2 Likes
4

Ok, if I understand correctly the resulting glTF is a pointcloud containing all the vertices from the SPZ (including all the attributes like scale, rotation and spherical harmonics), and the complete SPZ in a buffer? Seems like a lot of duplicate info here.

5

There is no duplicate data stored explicitly. The glTF with the KHR_spz_gaussian_splats_compression extension defines a mesh primitive like the following:

        {
          "attributes" : {
            "POSITION" : 0,
            "COLOR_0" : 1,
            "_ROTATION" : 2,
            "_SCALE" : 3,
            "_SH_DEGREE_1_COEFF_0" : 4,
            ...
          },
          "extensions" : {
            "KHR_spz_gaussian_splats_compression" : {
              "bufferView" : 0
            }
          },
          "mode" : 0
        }

It defines the attributes of the mesh primitive for all the properties that are required for a Gaussian Splat. And these attributes are standard accessor objects. Looking at one of these accessors, it is defined as follows:

    {
      "componentType" : 5126,
      "count" : 786233,
      "type" : "VEC3"
    },

Note that this accessor does not define a bufferView. Instead, there is that extension object in the mesh primitive:

            "KHR_spz_gaussian_splats_compression" : {
              "bufferView" : 0
            }

This one points to a bufferView. And this buffer view just contains the raw SPZ data.

The idea here is that the mesh primitive defines the attributes (as accessors), but these accessors do not define ~“where their data is stored”. And the data is stored (in SPZ form) in the bufferView that is defined in the extension object.

Note that this is allowed as of the glTF core specification, exactly for the use case that is described here: It should be possible for “fill” accessors with data that is not defined with a buffer view, but is coming from an extension. In fact, the same approach is used for KHR_draco_mesh_compression: It also uses accessors without buffer views, and they are filled with the data that is unpacked from the the Draco-compressed data that is stored in a dedicated buffer view.

There are some points worth mentioning here:

  • In theory, the glTF loader/viewer does not necessarily have to fully uncompress the data on the CPU. For example, SPZ stores the _SCALE property in 8-bit quantized integers, but the extension defines the accessors to be float. And the loader/viewer could upload that quantized data and perform the dequantization in the shader. (The accessor rather defines how the data is interpreted in this case, as some sort of ‘interface definition’ for the renderer)
  • The current definitions of the attribute names will likely still change. Right now, they are _SCALE. But for reasons laid out in the discussion about splats in glTF, this might end up to be something like KHR_gaussian_splatting_SCALE or so.
1 Like
6

Once more the disclaimer:

The following is unrelated to Cesium.

I’m just a nerd who cannot set the right priorities.

I noticed that there is not much tooling for Gaussian splats in Java, so I just created JSplat. There’s not yet so much to see: It only offers readers/writers for splat data in ‘gsplat’, PLY and SPZ format, as well as glTF with KHR_spz_gaussian_splats_compression extension. (The latter makes limited sense in its current form: It can only handle glTF files that contain a single splat primitive. But maybe there will be additional tooling around that in the future)

Creating this library involved releasing JSpz into Maven Central, as well as publishing and releasing Ply - a PLY reader/writer for Java that I started ~15 years ago, and that had been quietly catching dust in my ‘Develop’ folder until now…

2 Likes