How to integrate Gaussian Splatting with traditional 3D tiles for simultaneous rendering?

CesiumJS
1

I’ve combined 3D Mesh and Gaussian Splatting (GS) in a tileset, however, it’s not rendering correctly. The 3D Mesh failed to parse after adding the following data.
“extensions”: {

"3DTILES_content_gltf": {

  "extensionsRequired": \[

    "KHR_gaussian_splatting",

    "KHR_gaussian_splatting_compression_spz_2"

  \],

  "extensionsUsed": \[

    "KHR_gaussian_splatting",

    "KHR_gaussian_splatting_compression_spz_2"

  \]

}

},
Does it support co-rendering both types? If supported, what is the recommended implementation path?

2

Welcome to the forum @LoongZP!

It’s hard to help without more information, could you provide a Sandcastle that reproduces this issue? (preferably create a new API key for just the relevant assets and you can share with me via DM if they are sensitive)

Could you also share any errors you are seeing in the console?

If I understand correctly, you have a 3D mesh and a 3DGS in the same tileset? I am not sure if this is supported (I will check), in the mean time, is there a reason you are combining them into a single tileset? Have you tried keeping them in separate tilesets?

3

Yes,I have a 3D mesh and a 3DGS in the same tileset. I’m doing this to combine several tilesets into a single one for a ‘better’ user experience. To be honest, it’s just a PM requirement that I don’t really get either.

4

This is currently not supported. I pointed that out quite a while ago, and I think that there are very reasonable use-cases for this. But I now opened a dedicated issue at Cannot combine splat- and non-splat content in one tileset · Issue #13312 · CesiumGS/cesium · GitHub to keep track of this.

6

It is achievable if you need to modify the source code. I have tried adding both 3D Gauss and b3dm at the same time, and successfully loaded them by making simple code changes.

PixPin_2026-03-23_12-49-14
PixPin_2026-03-23_12-49-141393×742 327 KB

7

That’s great to hear! CesiumJS is open source so it would be a great contribution to create a pull request here with your modifications and link Marco’s issue on it.

Cheers!

8

At the first glance, it is relatively easy to “fix” this. I already did that, as shown in the screenshot in the follow-ups-issue. And I think that change was really something trivial, like adding some if (!defined(something)) return; at the right place.

But there are some further things to consider here: The splat rendering currently makes assumptions about the structure of the glTF, and these assumptions are plainly wrong (in most cases). Specifically: You could have “splat primitives” and “mesh primitives” in one glTF asset. And this will still not work with a “simple” fix.

So…

  • The linked issue currently says that it’s not possible to mix splat- and non-splat glTF assets in one tileset.
  • I’ll probably open another issue, pointing out that it’s not possible to mix splat- and non-splat primitives in one glTF asset.

And I’d add mutual links between these issues, because I think that if something is done about that, then it should be a change that fixes both issues at the same time. This may involve a larger refactoring, but I think that this is something that has to be addressed more holistically. There is even one layer above that, namely that you cannot combine two splat-tilesets in one scene. This is already tracked in Incorrect depth sorting between Gaussian models at different heights in the same area · Issue #13256 · CesiumGS/cesium · GitHub , and may also be related. Ideally, there would be a solution for all three issues, but eventually, this may boil down to a re-write of much of the splat rendering functionality.

(Iff the case to mix splat- and non-splat content in a single tileset turns out to be critically important, then maybe a small PR with a bandaid fix could be OK. It would take 10 minutes, and “make things work” on a shallow level. But a robust solution for mixing different content types will require more thought)

9

You’re right :+1: . I’ve considered some of these points as well, but not as comprehensively as you have. Therefore, what I’m describing here refers to 3D Gaussian primitives and b3dm primitives, which is the simplest implementation approach I can think of for now.

10

I’m not opposed to adding a “quick fix” to make it possible to combine splat- and non-splat data in one tileset. (You mentioned B3DM, but I assume that this refers to ~“all other types”). As I said, I think that the fix was really simple - something like if (thisThingIsNoSplats) { return; }, which should apply to B3DM as well as I3DM/PNTS/CMPT, and glTF without splats.

The other case (meshes and splats in one glTF) can then be tracked in that issue that I still intend to open (and the case of multiple splat tilesets in one scene already is tracked).