Question: Clarification on rendering ADD refinement tile children

3D Tiles
1

Hello! I’ve received some requests about the differences in the number of tiles loaded between Cesium and the 3DTilesRendererJS project and wanted to clarify what the “right” thing to do is. I’ve made some comparison projects and have confirmed that Cesium is, indeed, loading fewer tiles in a tile set with an “ADD” refinement root and 256 child tiles provided by an end user – specifically this tile set structured like so:

root
refine=ADD
geometricError=500
 └ 256x children
   geometricError=1

The section on “refinement” in the 3D Tiles specification states the following:

If the tile has replacement refinement, the children tiles are rendered in place of the parent, that is, the parent tile is no longer rendered. If the tile has additive refinement, the children are rendered in addition to the parent tile.

And in the geometric error section:

If the introduced SSE exceeds the maximum allowed, then the tile is refined and its children are considered for rendering.

My reading of this is that if an “ADD” refinement tile is “refined” then its children should render (barring any frustum or visibility culling, etc). However when a child of an “ADD” refinement tile is encountered in Cesium it seems to use the parent tiles geometricError value to calculate the child tile’s screen space error with the child’s bounding box and determine it to be visible or not. However it’s not clear from the comments of the code why this is done and is not following the described purpose of geometric error in the specification.

So my question is - what’s the rationale behind using the parent’s geometric error this way? And how can this behavior be reconciled with the spec? What’s the right thing to do? I’m hoping there’s a “canonical” correct rendering of a tile set so a loaded model can be reliably rendered across applications.

Thank you!

cc @sean_lilley @Gabby_Getz

1 Like
2

Given that this should preferably be aligned with cesium-native/Cesium3DTilesSelection/src/Tileset.cpp at 70bd62a9f4883d26def3d1dddd08c6a67217ec86 · CesiumGS/cesium-native · GitHub etc, also tagging @Kevin_Ring here.

1 Like
3

cesium-native doesn’t do anything like this. It’s an interesting optimization, though. I could be convinced we should add it (and probably add a note about it to the spec as well).

Here’s how I think about it… The SSE of a tile determines whether that tile has enough detail (the SSE is below the max), or whether we need to show child tiles instead in order to get more detail. Normally (in cesium-native, and according to the 3D Tiles spec), when the parent tile doesn’t have enough detail, we show all the (non-culled) child tiles. But some of those child tiles may be very close to the camera, and some may be very far away. This is especially true in extreme cases like your tileset with a parent tile that has 256 children!

So this optimization in CesiumJS essentially says: ok, the parent tile as a whole doesn’t have enough detail. But this particular child tile is really far away from the camera. Maybe that far away section really does have enough detail, even if the closer bits don’t, because it’s so far away. What’s the parent tile’s SSE in the region of that child tile? If that is lower than our max, then it might be fine not to render this additive-refined child tile.

IMO that is against the letter of the spec, but it’s a reasonable thing to do. It will make the distance at which that child tile content appears more uniform, for one thing. No matter which direction you approach the child tile from (i.e., from “over” other sibling tiles, or from outside the parent entirely).

2 Likes
4

Here’s how I think about it…

Thanks - this is roughly the conclusion I’ve come to, as well, but I’m curious as to which other cases are handled this way. The linked function early-outs if refinement is not REPLACE or a “non-geometry” tile rather than only applying it to additive refinement tiles. Does this mean there are other conditions under which this logic is used?

This change also seems to replicate the behavior you’d see if there were an intermediate node between the “ADD” parent tile with and the geometric children that have the same bounding box as the child and the parents geometric error value:

root
refine=ADD
geometricError=500
   ├ empty child (1/256)
   | refine=ADD
   | geometricError=500
   |    └ geometric child
   |      geometricError=1
   └ ...remaining children

I’m torn between forcing out-of-spec behavior that is otherwise already possible to achieve if a tile set is structured well. At some point it’s encouraging what should be considered poorly-formed or poorly-optimized data sets relative to the spec. I’ve had users come to the project with files that have tens of thousands of children under a REPLACE node, for example, asking why it works in Cesium while it’s not loading / loading slowly in 3DTilesRendererJS project because of implementation choice made in Cesium that specifically applies to REPLACE root nodes enables it to run (see this issue).

IMO that is against the letter of the spec, but it’s a reasonable thing to do

I’m fine with adding it if this is going to be become a common optimization but it would be good to know where some of these Cesium-specific behaviors fall in terms of “canonical 3d tiles rendering behavior”, “pure optimizations with no visual sacrifice”, and “optimizations or choices that don’t align with the spec”. It would nice they were documented so users can be aware of creating a tile set that may rely on non-standard behavior. The spec should at least be changed to call this out as a reasonable “non normative” implementation if it’s going to be kept since users look at CesiumJS as “correct” behavior for 3d tiles and complain if it’s not matched.

edit

Other features from the Unreal like “an external tile set should always refine” resulting in the geometric error being ignored on external tileset also doesn’t seem to follow the spec, either (CesiumJS and 3DTilesRendererJS do roughly the same thing, it seems). At this point tile sets have been generated that rely on this behavior, though. Or at least this may be an ambiguous point in the spec? It’s something that could be clarified, I think.

1 Like
5

(I think that some places in the previous answer use ‘REPLACE’ where ‘ADD’ was intended :thinking: )

I cannot say much about the specific implementation in CesiumJS. From what I think to have understood, it sounds like a reasonable optimization: If one child that is supposed to be ADDed does not really add any value (in terms of visual quality), then it is omitted by CesiumJS.

IMO that is against the letter of the spec,…

One could argue about that. (And conversely, one could either complain that the spec is “not strict enough” or point out that it (intentionally) “leaves room for custom optimizations”).

The spec says that…

“…the tile is refined and its children are considered for rendering” and not
“…the tile is refined and its children are rendered (‘unconditionally’)”

There are many reasons for why a certain tile is not rendered (summarized as “culling”), and one could say that these reasons - together with the optimization for the ‘ADD’ case here - fall into the category of implementation choices.

External tilesets have their own whole category of open questions. I wasn’t aware of the explicit special handling in cesium-native. But in context of the recent discussion about the geometric error, there’s a TODO on my list to look up what the behavior is in CesiumJS in this case. When there is a “leaf” tile with a geometric error of 100, that contains an external tileset with a root geometric error of 1000, then all the refinement rules fall apart.

(I thought that the “runtime value” of the geometric error of an external tileset might have to depend on the geometric error of the node that they are attached to - just to ensure that there is a strictly decreasing progression of the geometric error values when traversing from the root through a ‘leaf’ into an external tileset. But that’s certainly not specified, and it’s probably easy to come up with cases where it wouldn’t make sense either…)

6

(I think that some places in the previous answer use ‘REPLACE’ where ‘ADD’ was intended :thinking: )

I’ve just reread my comment and it’s correct. If you’re referring to this line:

I’ve had users come to the project with files that have tens of thousands of children under a REPLACE node, for example, asking why it works in Cesium while it’s not loading / loading slowly in 3DTilesRendererJS project

Then yes, a user provided a file with a REPLACE node that had thousands of immediate children. Cesium’s implementation is such that even when an empty root node is set to REPLACE refinement it treats children as though they are under an ADD node and they get frustum culled and not loaded so you’ll get gaps when turning the camera. You can read this issue I linked previously for more details.

There are many reasons for why a certain tile is not rendered (summarized as “culling”), and one could say that these reasons - together with the optimization for the ‘ADD’ case here - fall into the category of implementation choices.

Typical culling strategies (frustum, occlusion) do not impact the resulting look of the final model. Reading “considered for rendering” as allowing runtime applications to randomly discard half the tiles on screen, additive or not, and still consider it representative of the file and adhering to the intent of the format rather than as referring to the culling methods described in the spec I don’t feel is a helpful interpretation. It would be nice to be able to harmonize on a common, “canonical” rendering of 3d tiles so applications can reliably display the same thing and deltas from that can be deliberate choices. glTF specifies the PBR shading model to be used, for example, so the intended look is clear but applications can choose to replace the material with Phong shading as an optimization if they choose. Leaving something like this so open for interpretation in a specification makes this difficult and it would be great to hear from @Gabby_Getz or @sean_lilley since CesiumJS seems to be the only place doing this.

I wasn’t aware of the explicit special handling in cesium-native .

If external tile sets are treated as “empty nodes” that can stop traversal then you’ll potentially wind up with gaps unless the external tile set node has a geometric error high enough to “immediately refine” but even then you could have a gap while the subtree geometry loads. Treating “subtree nodes” as being “replaced” by the root node of the loaded subtree (or multiple subtrees) rather than children of an otherwise non-renderable node means there are no holes but also means the geometric error of an external tile set node is effectively useless. The spec doesn’t specify this behavior but is loose enough to interpret it this way. It would be good to clarify what the right thing to do is in the specification.

just to ensure that there is a strictly decreasing progression of the geometric error values when traversing from the root through a ‘leaf’ into an external tileset. But that’s certainly not specified

The spec says “Generally, the root tile will have the largest geometric error, and each successive level of children will have a smaller geometric error than its parent, with leaf tiles having a geometric error of or close to 0” so it’s not strictly specified that geometric error must decrease, which may be strange but it’s okay, in my opinion.

1 Like
7

Right, I misinterpreted the context of the REPLACE there. I should have read that (and looked at the linked issue) more closely.

There may be additional culling criteria, where the engine finds that a tile may not be completely (“mathematically”) invisible, but would add little to the visual appearance, and therefore decides not to render it. For example

  • Fog based culling
  • Screen location based prioritization/culling
  • Screen coverage based culling

I cannot say with absolute certainty which of these criteria are implemented in CesiumJS. This may be due to my lack of familiarity with these parts of the code. It may also be because these culling criteria are often not made explicit. There is no code part where you have a

for (CullingCriterion c : cullingCriteria) {
    if (c.appliesTo(tile)) discard(tile);
    else render(tile);
}

and just look at each implementation. Instead, these criteria are usually “smeared” over the traversal process, with a bunch of variables and if-checks. Specifically:

  • dynamicScreenSpaceError and associated variables and checks seem to be (similar to) what could be a FogBasedCullingCriterion
  • foveatedScreenSpaceError and associated variables and checks seem to be what could be a ScreenLocationBasedCulling

And I think that there is no explicit form of screen coverage based culling. But if I understood this correctly, then the behavior of the ADD-refinement here could be seen as a form of coverage-based culling criterion: The child tile is small and far away (thus, covering only a tiny area of the screen), so it’s not worth to really render this.
(That’s what I derived from the summaries here. I could not derive this from the /** Documentation */ of the meetsScreenSpaceErrorEarly function… …)

All that said:

I agree that there are places in the specification where the intended behavior in terms of traversal should be made clearer. This could mean that the specification becomes more restrictive (which can be difficult). But even if some aspects are left to the implementation, then this should be made explicit. This could/should include ‘Implementation Notes’, about how implementations can handle certain corner cases, or which degrees of freedom they do have, specifically (e.g. about omitting children in the ADD-case).

This also applies tol the questions about external tilesets (e.g. whether there should be some sort of “artificial” node be inserted, how to handle possible ‘gaps’, etc). And (related to that): How to handle the case of non-decreasing geometric errors. (I cannot say much more about this. Maybe someone who is more familiar with the specific implementations can chime in here).

There are several issues and threads revolving around all this (random examples are How to support placeholder tiles - empty tiles used for culling but not refinement · Issue #609 · CesiumGS/3d-tiles · GitHub or Clarification on refinement and child tiles , but many more). And it should not be necessary to reverse-engineer the CesiumJS code, which always bears the risk of people assuming that it is “correct” and “canonical” (which it may very well not be in many cases).

8

Out of curiosity, I looked at the commit history for when this was added. It was very early on:

9

As another piece of information for this thread, here is the rather detailed explanation of how Tile selection works in Cesium native:

It is lengthy but is written well and seems pretty precise. It appears to be without a lot of reference to the spec. or how it may conform to it. Hopefully, it is consistent with the code linked above and mentioned observations.

[I think Cesium wants to encourage a wider, more diverse geospatial ecosystem for 3d representations on the web which is also standards based. There is a lot of value in well defined, tighter standards since this is a way to build trust for both users and implementers. On the other hand a lengthy narrative on Tile selection such as above is extremely useful but too expansive for spec. language. It is a difficult balance.]

1 Like