Tiles not loading accumulates

Many posts similar but I think this is new for the Community. We have 8 unreal 4.27.2 apps running the 1.18 cesium release that show the same error randomly. Over time using the app tiles will slowly drop out and stop loading. Once a tile stops loading it is doomed and never loads again. The tile can be very specific and be one of the LOD levels. In the linked video you can see different zoom levels with different tile dropouts. Refreshing the tileset does not refresh whatever lost “pointer” exists and the same tiles are not restored. The Output Log just shows “done” when refreshing the tileset. There are no errors.

Is there a more verbose log mode? Is there any way to debug this? Could this be due to a network transaction timeout and if so could cesium retry or at least report this? Maybe setting up a local tileserver would help but if we are chasing a unhandled timeout this seems precarious.

Hope this video link works for you:

Bumping this one. @Kevin_Ring What’s the best way to debug or log the loading of individual tiles?

There’s no “extra info” mode, so your best bet is to step through the code to see what’s going wrong. It could be along the lines of the web server returning a success code but a bad response, which then gets cached. A proper error response fro the server should not get cached.

Hi @Kevin_Ring ,

we have the same problems now for a few months. When running our flights simulator (UE 4.27.2) after a while some tiles start to disappear and are never loaded again, flying away from them till they are not visible anymore and flying back didn’t help, either. When a tile is gone it stays gone (so maybe this indicates that CESIUM reneder thinks the tile is still there, cached, so no need to load it again?).

The only way to get out of this is to stop the simulation and restart it again. And that’s bad for ongoing test flight sessions, wastes times and invalidates running sessions so they need to be repeated, so it’s a really bad issue.

The workstation used has 64 GB of RAM, a RTX4090 GPU, a i9-13900K CPU and s superfast internet connection to the CESIUM Ion server, so I’m quite sure it’s not a bug caused by insufficient memory or bad performance on our side.

Any help with this would be appreciated.

Best
Andreas

Hi @Lildreas,

Does refreshing the tileset help in your case? It’s a button on the Tileset in the Editor, or a method you can call from Blueprints.

In @cevanno’s case, it apparently didn’t help, which is quite perplexing. There’s very little persisted between refreshes, other than the request cache (which is stored on disk). But if the request cache were somehow poisoned, then restarting the app wouldn’t help either; you’d have to delete the cache.

Can you pin it down to a particular Cesium for Unreal version where it first broke? That might help us identify the cause. And of course if you can reproduce the problem in a minimal project and provide me with instructions for doing the same, that would be a huge help.

Thanks,
Kevin

Hi @Kevin_Ring,

thanks for the fast reply! I just stopped working on the sim for this year, but beginning of next year I will test this by implementing a hot key to refresh the tileset during runtime. So as soon as we encounter a problem we can hit the button - and hope that it helps.

While this might be a workaround to for the issue it doesn’t remove the cause. But here I fear I cannot help much, as I don’t know exactly when we first encountered the problem - I normally update the UE 4.27.2 plugin as soon as I get the info about a new release, then I check the release notes for any breaking changes and then I just re-compile and re-deploy. All of this is kind of automatic, normally a few hours later I already forgot that I did this. :smiley:

Reproducing this in a minimal project would be hard, as the flight simulator is in fact a dozen software packages running on three connected workstations, so just to give you a stripped down version of the Unreal IG (Image Generator, the visual part of the flight simulator) wouldn’t help, as e.g. all the flying movement is driven from another workstation via UDP packages etc.

Maybe someone else having this issue can pitch in an provide some build to reproduce it?

Have a good Christmas time & a few nice cocktails in the sun while we are freezing our b**** off in the German winter,
Andreas