Cesium causes app to seize after random amount of time

So, we had run into some issues with our app randomly locking up on cesium levels. This only occurred on the servers we are running the app on. When we used a profiler, it showed there was a large spike in GPU usage and then it would drop to 0 and the program would freeze.

This only happens on levels running cesium tiles. The only way we were able to stop the freezing is to add the command line arg “-onethread”.

I am unsure why this worked but it did. It seems like maybe there is some thread that is causing issues but when it is executed sequentially it works fine.

Please let me know if there are any logs or additional info you might need.

Hi @TeraDan!

I’m sorry that you’re experiencing issues with the Cesium plugin. From a glance, I can’t tell what could be causing the crash, but we can try to help with more information:

  • What versions of Unreal Engine and Cesium for Unreal are you using?
  • What device are you running your app on? Desktop? Mobile?
  • What tilesets are you using? For example, Cesium World Terrain? Google Photorealistic 3D Tiles?
  • Are there any common factors that occur before the freeze? For example,
    • Does the freeze happen after a certain amount of time that the level has been open?
    • Does the freeze happen in a particular area of the world?
    • Does the freeze happen after certain user inputs?
  • When the app freezes, does it eventually unfreeze? Or does the program seem to be suspended indefinitely?

Please feel free to include screenshots or add anything else that I might have missed. I know this is a long list of questions, but it’s all in the name of giving more informed help. :smile:

Once we establish some key information, we can troubleshoot from there. Thank you!

  1. We are using 5.3.2
    *Not sure what version of cesium, whatever is on the marketplace
  2. We are running on a server and connecting via pixel streaming. The server is using a T4 GPU
  3. We were initially using the google api tileset but switched to Cesium World Terrain with OSM buildings to test but it didn’t have any impact.
  4. Only common factor that we could determine is moving quickly and erratically. There wasn’t a common interval when it would freeze. Happens anywhere in the world.
  5. It freezes until the unreal thread times out and then it crashes.

Hi @TeraDan,

Thank you for the detailed information! It’s hard to say what’s going wrong. Does modifying the Cesium3DTileset settings help at all?

In particular, you could try:

  • Reducing the Maximum Simultaneous Tile Loads
  • Setting the Maximum Cached Bytes to 0 (to disable caching)
  • Disabling Forbid Holes (if you have enabled it)

Let us know if those settings make a difference for you! Especially if one setting makes more of a difference than the others.

I implemented the above changes and disabled running on a single thread.

It seems like it does not play nice with pixel streaming.

I tested by using windows remote desktop and I was able to move around fine.

As soon as I launched pixel streaming, it crashed. So it may be some combination of those 2 tools not playing nice with each other.

Hi @TeraDan,

Thanks for your response, and for looking deeper on your end! This is really helpful information. I’ve written up an issue on Github based on your description; we’ll post updates to the issue and this thread after we investigate more deeply. :pray:

Great! Thanks for all your help!

Hey @TeraDan,

I’m afraid we still need more information to troubleshoot your issue. In particular, can you:

  1. Report the exact Cesium for Unreal version you’re using. When you open Unreal Engine, this is displayed at the bottom right corner of the Cesium window, like so:

  1. When Unreal Engine crashes or unexpectedly closes, we expect to see a log of what happened. Do you see this log – can you locate and share it with us? These logs often contain valuable information for diagnosing the issue.

  2. Along the same line, when this crash happens, do you see a call stack that shares what line the application crashes on?

No need to be afraid! I don’t bite hard :slight_smile:

Looks like we are using Cesium 2.4

No call stack, it just says that the thread timed out after waiting 120 seconds. I have attached the crash folder.

Oddly, There was no log file that I could find, only the info in the crash folder.

UECC-Windows-1C8835CE44FEFA97109E58B3C74B953D_0000.zip (50.7 KB)

Thanks for attaching the crash folder. It’s weird that there’s not much information on what went wrong :thinking:

Still, I searched for this error online and I think there’s something to try. What Graphics API is your project using? Does changing it to DirectX 11 (or perhaps other modes) make a difference?

When changing the graphics api, are there any other changes that I need to make? I tried testing it and the tiles won’t load at all after I set it to DX11.

That’s strange @TeraDan, I’ve been able to switch the graphics API just fine. Just to confirm, you’re setting this in Default RHI under Platform > Windows, right?

Does this happen for any of the other rendering modes? And, do you have any messages or errors in the log? I wonder if the machine is simply unable to handle it.

But in any case, I found an Unreal forum thread that might describe what’s happening here. It’s about a memory leak with pixel streaming that ultimately shows the error message that you received in the crash dump.

It looks like there were PRs to address this, but they appear to have been merged. However, there’s a comment in the thread that suggests this tends to happen with runtime-created meshes:

It appears that this crash rarely occurs when using a cooked scene, unless there are meshes created at runtime.

So I guess it’s a matter of understanding why creating these meshes is causing errors with pixel streaming… which I have not been able to find any resources on :grimacing:

Another forum thread suggests setting g.TimeoutForBlockOnRenderFence to a large number like 9999999, though I don’t know about the implications of it. But it might be worth trying?