I’m reading live weather data from an external source and converting to Cesium 3D tiles v1.1 point clouds, PNTS, as binary glTF 2.0 (glb) files and rendering with the latest Cesium 1.143.
Now, experimenting with Gaussian Splats, I’m generating the tiles as 3DGS with spz compression.
It seems that the gaussian splats start to disappear too soon as you zoom out, even though the geometric error is set very high (100000.0) for both PNTS and 3DGS tilesets. See the example images.
Does the geometric error in 3DGS control rendering the same way as with PNTS?
Top, left: close up of point cloud with multiple weather levels
Top, right: same view with GS and just three weather levels
Bottom, left: broad view point clouds of today’s weather
Bottom, right: zooming out showing disappearing GS
That “black hole” of the disappearing splats at the center makes me think that this is the same as Gaussian splats disappear based on distance, rotation, view size, and size · Issue #13304 · CesiumGS/cesium · GitHub . Does this round area of disappearing splats feel like it is ~“fixed in screen-space” when you are moving and rotating the view? Then this would be even more likely.
(The issue description contains a potentially trivial fix - in doubt, I’d just open a PR and see where that goes 
…)
Yes, the missing hole does seem to be fixed in screen space.
Also, it was a bit tricky getting the scale data to show GS in Cesium. Seems that setting y=237 is ideal (anything higher will not show any GS) where y is the quantized scale value stored during the spz compression.
So, with that y value, pick a scale min and scale max, then using the inverse log-encoding, calculate a scale value to be quantized:
scale = Math.exp(Math.log(scaleMin) + y*(Math.log(scaleMax) - Math.log(scaleMin))/255.0);
For example, set scale = 0.9522497, scaleMin = 0.5, and scaleMax = 1.0 will quantize to the value 237, and that seems to get the best results.
Then I’m reasonably sure that this is the same issue. (I added a note in the issue, and will try to create a PR soon, but no promises right now)
A bit of context:
The data set that you have there is “unusual” in some ways. It is HUGE in terms of the geographic extent. Usually, splats are used for … you know, some reconstruction of a toy that someone scanned with a smartphone, or a single building from a drone scan at best.
Covering a “nearly-global” extent does raise the question of what the proper size/scale of the splats should be. You can hardly pick a splat size that makes sense for both the “global” view and the zoomed-in view (say, the level of a city). When the splats are large enough to be sensibly displayed on the global scale, then they would just be huuuge, washed-out “gaussian blob planes” when zoomed in. And given that the scale from SPZ does indeed encode the scale as an 8-bit log-encoded value, a value of 237 means that the splat is thousands of meters large.
It may not immediately be obvious, but this is strongly related to the issue that you’ve been observing: Apparently, in the implementation of the Gaussian splat rendering in CesiumJS, someone added a check that boils down to this: “When the size of the rendered splat is smaller than a single pixel, then don’t display it”. And even though your splats are huge, when you zoom out to the “global” level, they start to become smaller than a pixel, causing them to disappear.
To be clear: This should not happen 
It’s a bug in the implementation. It’s not clear what the “perfect” solution would be. Imagine one splat being so tiny that it only occupies 1/1000th of a pixel - there is no point in rendering that (literally… no point …). But lowering the limit from “one pixel” to “one-eigth of a pixel” should already alleviate the issue.
Somewhat unrelated: I’m nearly a bit surprised that the data seems to work … I mean, at all. How exactly are you encoding the positions of the splats? The point is that in SPZ, the position of splats may - by default - only be in the -2k … 2k range (see Question about position encoding · Issue #52 · nianticlabs/spz · GitHub for some rabbit hole).
Did you tweak the fractionalBits when writing this data?
Generating the splat positions, first, I divide the conus into smaller tiles - the center of each tile becomes its reference point. Each tile, below, represents a glb with embedded spz GS.
When storing the position data, each lat, lng, alt point is converted to the ECEF (Earth-Centered, Earth Fixed) coordinate system and rotated appropriately.
Then, with these tile points, relative to the tile reference point, the maximum span is found leading to the fractionalBits of the tile:
fractionalBits = (byte)(23 - Math.ceil(Math.log(maxSpan)/log2)); where log2 is the natural log of 2.
For the current configuration of tiles and today’s weather, the fractionalBits vary from 4 to 13.
Finally, I use the glTF node.matrix transformation to translate the tile to its reference location.
That makes perfect sense. It was not entirely clear from the first screenshots how large one “tile” is. And some details certainly depend on the exact tileset structure. For example, one could consider to let this data set (i.e. the tileset) have some concept of ‘level of detail’. For example, you could have one ‘nation-wide’ tile with 100k large splats (where each splat may very well represent some form of ‘average’ for a certain area), and let this tile have actual child tiles (maybe as a quadtree) where each child tile also has 100k splats, but smaller and more fine-grained ones. This would allow for the usual geometricError-based refinement when zooming in. In any case, if you had one nation-wide tile, then the fractionalBits might become an issue (unless you explicitly accept that the positions of these splats may have some quantization error - but that would still make sense, because they’d be 1. only ‘averages’ to begin with, and 2. they’d immediately refine to the more precise ones when zooming in).
So to ask that directly: Do you already apply (or consider) a form of ‘level of detail’ for this data?
Yes, I do have branching set up by way of a multiplier array to reduce the max geometric error, where the position data is stored in an appropriate branch ‘bucket’ for processing into separate glb files.
The trick is finding a visually appealing pattern to determine which bucket to store a point into.
This case, I have 4 multiplying factors, starting with 1 (highest GE), and looping over lat and lng, compute the branch index like this: ibranch = 2*(ilat%2) + ilng%2;
In the image below, I had to reduce these factors quite a bit so you can see the LOD reduction before the GS disappear.
I don’t fully understand some aspects of these ‘factors’ or what the ‘(visually appealing) pattern’ refers to.
Maybe that depends on the data basis: I assume that you have an extremely fine-grained data set of observations, which could represent the highest level of detail. From there, there are different approaches for creating a tileset with LOD from that. You could slice that into a grid of X * Y “cells” which are the leaf files, and then compute one appropriate parent for each leaf tile (bottom-up). Or you could start with the full data set, and slice it into 2x2 cells, recursively, until each cell contains fewer than a certain number of points (top-down).
Each non-leaf-tile content would be a subset of the data that it originally contained, and one interesting aspect is how that subset is computed. Just picking a true subset of the actual splats will likely not yield nice results: The splats would be too sparse. So as you come closer to the root of the tile hierarchy, the splats will have to be larger.
I’ve been pondering with some of that. You may have seen the data sets from the issue. (Of course, these are purely artificial, but eventually, the question about the structure of the tileset is independent of where the “bottommost” splats are coming from). In that artificial data, I picked the size (scale) of the splats so that they nicely “fill” a certain region. And it was exactly that which led me to discovering that issue: On the leaves, the scale was too small, causing their rendered size to be less than a pixel.
Small update: I created a PR for fixing the premature discard at Reduce threshold for discarding small splats by javagl · Pull Request #13642 · CesiumGS/cesium · GitHub , but I think that there’s still something odd with the splat rendering in CesiumJS…
