We’re using a metallicRoughnessTexture with full metallic in water areas to try simulating reflection of the sky in water in our created geocentric 3D Tiles datasets from Remoscape. In Unreal and Unity the sky, and at least in Unreal even the other geometry, is reflected nicely. But CesiumJS seems to use some tilted blue/brown reflection map, so parts of our water appears brown, see below. Why is this, does it have to do with the geocentric coordinates? Can it be fixed?
It’s hard to be sure from just looking at the screenshot. But visually this might be related to New 1.123 Environment Map Rotates Together With Model · Issue #12310 · CesiumGS/cesium · GitHub .
Not sure whether threre’s a quick way to confirm this. Some brainstorming: If this is a tileset, one could consider some hack like
tileset.tileLoad.addEventListener(function(tile) {
const maybeModel = tile._content?._model;
if (maybeModel) {
model.referenceMatrix = Matrix4.clone(Matrix4.IDENTITY);
}
});
or so, just to see whether that yields the expected result, which might be a hint at how that issue might be fixed, eventually…
Thanks for the suggestion, but it doesn’t seem to help unfortunately. I tried a couple of other things from the thread you linked but to no avail. It doesn’t seem to matter what I set referenceMatrix to.
My dataset also uses “gltfUpAxis”: “Z”, which I thought might be related, but I tried just removing it and it didn’t really help. I guess I would have to transform the coordinates accordingly in that case, but since nothing happened whatever I set referenceMatrix to, I don’t know…
I have not yet investigated all details of the issue, but in this comment I mentioned that instead of using the referenceMatrix in this line of the code, using context.uniformState.enuToModel seemed to fix it, but with the huge disclaimer: There are a lot of matrices, and their meaning is often not … documented extensively … so this was, admittedly, the result of somewhat randomly poking around in the code.
I’m not sure whether 1. you could try this change locally (even if it’s a long shot), or 2. share the data set where you observe that issue, either here of in the CesiumJS repo issue so that it may be investigated further. Both could be valuable steps towards finding a proper solution for the issue (even though there doesn’t seem to be a specific timeline for that right now).
Until I get the possibility to try your change locally, here is a similar smaller dataset showing the same problem:
Thanks, it indeed looks like another case of this issue. I left a comment at New 1.123 Environment Map Rotates Together With Model · Issue #12310 · CesiumGS/cesium · GitHub
Great thanks! I’ll see if my customer can change this locally or if we’ll have to wait for a fix from Cesium.
I’ll see if my customer can change this locally
If that refers to that code change of using context.uniformState.enuToModel, then I should emphasize that this is not a “proposed solution”. It was only something that apparently “happened to work”, and it has to be checked in how far that makes sense.
(That enuToModel matrix actually depends on the view, and not on the model itself, so it’s unlikely that this is the right solution. But it may be a hint that the ENU-to-fixed-frame transform of the model position will have to be taken into account there, somehow…)
In terms of other quick fixes or workarounds: When loading a single model, then it may be possible to set the referenceMatrix manually.
(With the “tiny” ( 
) degree of freedom: To which value?)
But when these models are part of a tileset, then it’s more tricky. (And I did spend a bit more time than I’d like to admit with zooming into that)
The first question is: What is actually wrong there? And the possible answers are
- the value of the
referenceMatrix - using the
referenceMatrixin that computation - both of the above
- none of the above
- something completely different
Given that the meaning and purpose of the referenceMatrix is not clear for me, it’s hard to say what is the case here. Looking at the code, one can see that the referenceMatrix is set to be the tileset.clippingPlanesOriginMatrix in each frame. (What do clipping planes have to do with image based lighting? Probably nothing. But that’s the way it is…). The clippingPlanesOriginMatrix is also updated in each frame, based on the _initialClippingPlanesOriginMatrix. And this one should be initialized to be the ENU-to-fixed-frame matrix at the center of the tileset. But it isn’t… at least, not really. It is supposed to be initialized in this line, but this line is not executed here. The initial bounding sphere that is computed there does not take into account the root transform (!) of the tileset, meaning that the originCartographic.height there is -6371450.300484946.
So… lots of matrices here, and possible reasons for the issue: It might be that this particular issue is not even caused by the linked issue, but rather by a wrong computation of the _initialClippingPlanesOriginMatrix. (Even though this should have nothing to do with image based lighting after all…).
However, a tl;dr:
The following seems to cause the tileset to be displayed correctly. But this is also not a recommended solution, obviously. There are quite a few things to be sorted out here…
const viewer = new Cesium.Viewer("cesiumContainer", {
globe: false
});
const tileset = viewer.scene.primitives.add(
await Cesium.Cesium3DTileset.fromUrl(
"http://localhost:8003/tileset.json", {
debugShowBoundingVolume: true,
}
)
);
const matrix = Cesium.Transforms.eastNorthUpToFixedFrame(
tileset.boundingSphere.center,
Cesium.Ellipsoid.WGS84,
new Cesium.Matrix4(),
);
// XXX Private variable access!
tileset._initialClippingPlanesOriginMatrix = matrix;
const offset = new Cesium.HeadingPitchRange(
Cesium.Math.toRadians(-135.0),
Cesium.Math.toRadians(-22.5),
8000.0
);
viewer.zoomTo(tileset, offset);
Wow, nice! It seems to work fine. I realize this is not the final solution, but at least it hopefully makes my customer’s demo next week work better for now. Thank you Marco, great work! I hope this will bring focus to the issue and that it will be properly fixed soon.
I hope that demo does not also involve clipping planes In any case, I’ll bring up whether this should be tracked as a separate issue (because I think that _initialClippingPlanesOriginMatrix is wrong, but that still has to be confirmed), and ask about possible timelines for addressing the issue(s).
I actually tried using a ClippingPlaneCollection to cut the Cesium globe around this model and it works fine as well. I can understand though if it hadn’t worked, since I modified the clipping plane origin matrix…
However, another dataset still behaves weirdly and I don’t know if it’s the same problem originally. It’s water is always brown and I can’t tell if the reflective map is tilted there as well, or if it’s something else. The dataset has asset ID 3385854 in my Cesium ION account, I don’t know if you can reach it?
I cannot directly access user data sets on ion (only ~“an admin” can do that, way beyond my pay grade). But in any case, it’s hard to say who can allocate the time to figure out what’s wrong there. (What I did there was also just poking around in areas of the code that I’m otherwise not familiar with, just to see whether it’s something “obvious”. Actually doing the math and finding the definite reason and solution may take more time…)
However, you could create a dedicated token that allows accessing only this asset/data set, and send that token to me via PM. Then I could have a look. But given the complexities and the many possible reasons and possible solutions, I cannot make promises…
Here is the dataset showing the brown reflection;
It fails at least for me when I’m using:
viewer.zoomTo(tileset,new HeadingPitchRange(0.0, -0.5, tileset.boundingSphere.radius * 2.0));
It works when using this instead:
viewer.scene.camera.setView({
destination: new Cartesian3(3265356.1840435807, 970206.4467962357, 5375924.863580823),
orientation: new HeadingPitchRoll(3.761904361783925, -0.41711651024004737, 0.00011987798853496656),
});
Strange, huh?
In our private discussion, me and Marco got differing results that seem to depend on the current timing or something. Calling setView instead of zoomTo seemed to work better in some cases.
But I have come across another model here (much larger so cannot share it) where this didn’t matter. And it shows even more weird stuff, apart from the reflection map being brown it also as these strange blue spots here and there, see below:
Does anyone have more ideas of how to fix this?