We’re running Unreal as a library, which means a main application with Unreal linked and running within one of the application’s windows (GUELibraryOverrideSettings.bIsEmbedded is true). This means the main application is in charge of calling UE’s Tick function instead of UE being responsible for ticking itself. Our challenge is to load Cesium assets without calling the Engine’s tick function.
We were able to get Cesium3DTilesets to load by manually ticking each Cesium3DTileset (and only the Cesium3DTilesets) regularly on the game thread, via AsyncTask(ENamedThreads::GameThread, ..) and FTaskGraphInterface::Get().ProcessThreadUntilIdle(ENamedThreads::GameThread); We are not calling the Engine’s Tick().
However, when we attempt to unload the map, we get garbage collection errors and a crash. The garbage collection errors look like this:
LogLoad: Error: Previously active world /Game/test.test not cleaned up by garbage collection!
LogLoad: Error: Once a world has become active, it cannot be reused and must be destroyed and reloaded. World referenced by:
Looking for existing references to World /Game/test.test...
(PendingKill) (async) CesiumGltfComponent /Game/test.test:PersistentLevel.Cesium3DTileset_11.None
During map change, the Engine makes it to UCesium3DTileset::EndPlay(), but not to UCesium3DTileset::BeginDestroy() or UCesium3DTileset::IsReadyForFinishDestroy()
How can we properly load and clean-up Cesium3DTilesets without ticking the Engine?
It’s hard to tell exactly where the error is coming from based on those lines - do you have more of the log to share? From a distance I’d guess that since BeginDestroy isn’t being called, ResolvedGeoreference isn’t cleaned up on ACesium3DTileset before switching levels, so Unreal can’t garbage collect the object. Have you tried manually calling InvalidateResolvedGeoreferenceas BeginDestroy does?
How can we properly load and clean-up Cesium3DTilesets without ticking the Engine?
I think the short version is that you cannot. Cesium for Unreal assumes that Actors, Components, and other game objects will be ticked. I would think that many things in Unreal would expect this, and would not work correctly if it’s not true.
The specific missing Tick that is probably causing problems for you in this one:
We don’t have any experience with your specific use-case (bIsEmbedded=true), and you’re the first person we’ve heard of using it, so it’s not something we know much about. We’re happy to accept reasonable pull requests that make Cesium for Unreal work better in this context, though!
LogWorld: UWorld::CleanupWorld for testMap, bSessionEnded=true, bCleanupResources=true
LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated
LogStreaming: Display: 0.014 ms (0.005+0.009) ms for processing 33/177 objects in NotifyUnreachableObjects( Queued=0, Async=0). Removed 7/7 (130->123 tracked) packages and 26/26 (164->138 tracked) public exports.
LogUObjectHash: Compacting FUObjectHashTables data took 0.43ms
LogLoad: Error: Previously active world /Game/Project/Maps/Test/testMap.testMap not cleaned up by garbage collection!
LogLoad: Error: Once a world has become active, it cannot be reused and must be destroyed and reloaded. World referenced by:
LogLoad: Looking for existing references to World /Game/Project/Maps/Test/testMap.testMap...
(PendingKill) (async) CesiumGltfComponent /Game/Project/Maps/Test/testMap.testMap:PersistentLevel.Cesium3DTileset_11.None
... (repeats) ...
(PendingKill) (async) CesiumGltfComponent /Game/Project/Maps/Test/testMap.testMap:PersistentLevel.Cesium3DTileset_0.None
(PendingKill) (async) CesiumGltfComponent /Game/Project/Maps/Test/testMap.testMap:PersistentLevel.Cesium3DTileset_0.None
... (repeats for each tileset) ...
LogReferenceChain: Referenced by 5 more reference chain(s).
LogWindows: Could not start crash report client using ../../../Engine/Binaries/Win64/CrashReportClient.exe
LogMemory: Platform Memory Stats for Windows
LogMemory: Process Physical Memory: 747.56 MB used, 860.93 MB peak
LogMemory: Process Virtual Memory: 1927.84 MB used, 2044.05 MB peak
LogMemory: Physical Memory: 27820.32 MB used, 37591.93 MB free, 65412.25 MB total
LogMemory: Virtual Memory: 32195.50 MB used, 47097.23 MB free, 79292.73 MB total
Message dialog closed, result: Ok, title: The UE5-Game Game has crashed and will close, text: Fatal error!
LogWindows: FPlatformMisc::RequestExit(1)
LogWindows: FPlatformMisc::RequestExitWithStatus(1, 3)
LogCore: Engine exit requested (reason: Win RequestExit)
I created a callback for FWorldDelegates::OnWorldCleanup in each ACesium3DTileset, then call InvalidateResolvedGeoreference() within that callback, but still get the same errors when loading new map.
There were some issues with GUELibraryOverrideSettings.bIsEmbedded, but nothing major. Epic implemented this flag with the idea that the Engine would not tick unless the external app directed it.
I had a similar idea with the AmortizedDestructor. As a test, I made a map with one ACesium3DTileset, then call the following code at the end of ACesium3DTileset::BeginPlay(). Not pretty, but meant to be a test. I verified that ticks occurred on each of the objects.
Async(EAsyncExecution::ThreadPool, [this]()
{
for (int32 i = 0; i < 10; i++)
{
FTaskGraphInterface::Get().ProcessThreadUntilIdle(ENamedThreads::GameThread);
AsyncTask(ENamedThreads::GameThread, [this]()
{
if (!bShuttingDown)
{
for (TObjectIterator<UObject> iterator; iterator; ++iterator)
{
UObject* object = *iterator;
if (ACesiumCameraManager* toTick = Cast<ACesiumCameraManager>(object))
{
toTick->Tick(0.125);
}
else if (ACesiumCreditSystem* b = Cast<ACesiumCreditSystem>(object))
{
b->Tick(0.125);
}
else if (ACesiumGeoreference* c = Cast<ACesiumGeoreference>(object))
{
c->Tick(0.125);
}
}
CesiumLifetime::amortizedDestructor.Tick(0.125);
getAssetAccessor()->tick();
FTSTicker::GetCoreTicker().Tick(0.125);
Tick(0.125);
}
});
FPlatformProcess::Sleep(0.125);
});
If this line is commented in ACesium3DTileset::Tick(), then it does not crash (but also never finds a tile to load): _pTileset->updateView()
I can uncomment that line and reload the map without crashing as long as updateView() doesn’t return tiles in its results.
That tick in AmortizedDestructor needs to keep happening until the object’s IsReadyForFinishDestroy returns true. That involves at least the following:
The HTTP thread needs to be able to continue processing requests and dispatching the corresponding callback. The HTTP thread only dispatches to the game thread (unfortunately), so if you block the game thread, it won’t happen.
The renderer thread needs to release the corresponding RHI resources. I don’t know what’s might be needed to ensure this part actually happens.
There may be other things. UObjects can implement IsReadyForFinishDestroy to do whatever async destruction work they need; it’s a bit of a black box.
But blocking the game thread and calling Tick repeatedly is unlikely to be sufficient. All that will do is repeatedly check IsReadyForFinishDestroy and find that it’s still returning false.
By the way, we reason we have all this complicated cleanup code is because Cesium - by necessity - creates and destroys many UObjects while it’s running, as resources are streamed in and out based on the camera view. If we didn’t do some manual cleanup, a) the Editor would quickly run out of memory and crash, because Unreal’s garbage collector is - frustratingly! - disabled when running in the Editor, and b) memory usage in-game would be much higher because resource cleanup would be at the convenience of the garbage collector, which runs relatively rarely.