I’m currently using cesium in UE5.1 with google earth’s photogrametry 3d tiles but I’m having an issue rendering out a sequence.
My issue is that when I hit render using movie render queue inside sequencer the render comes out with either tiles popping in and out or some not loading at all, I saw in this post (Unable to reliably render sequence to file - #5 by BryceKaufmanTPS) that supposedly unreal should wait for the tiles to fully load but that’s not currently the case.
If I’m just in the editor or in Play in Editor mode and I keep the camera still until everything fully loads it’s looking fine so I don’t believe it’s an issue with lack of memory in the computer or something like that, it just takes a little while to load everything up and when rendering Unreal doesn’t wait for everything to be fully ready before saving the image so every frame just ends up half-loaded.
Is there some obvious setting that I’m missing?
Hopefully someone can help out with this issue, thanks in advance!
We’ve periodically had reports like this, but so far haven’t been able to reproduce the problem. See here:
If you can give us step-by-step instructions for reproducing the problem - preferably by starting with the Cesium for Unreal Samples project - we can investigate.
The most likely cause is that Cesium is using the “wrong” camera to select tiles to render your movie. But it’s not clear when and why that happens.
I downloaded the Cesium for Unreal Samples project and was able to recreate the issue there, here’s how:
I opened the project, enabled the Movie Render Queue plugin and restarted the editor.
I opened it again and went to the 02_CesiumMelbourne map.
I created a new Level Sequence actor and created a Cine Camera Actor which I animated moving from up in the sky to closer to the city.
Then I rendered through movie render queue with the following settings:
The issue is not as noticeable as in the renders from my other project, I’m guessing because there are fewer things in this project so it’s able to load faster but you can see that the engine is not waiting for everything to be fully loaded before saving the frame.
Here are some screenshots from my project where the issue is more obvious.
Frame 40 of my project, you can’t really notice much here but it’s because I added 800 warmup frames before the render starts so everything that’s visible at the start has as much time to load as possible:
Frame 96, the camera moved closer to the ground so the tiles in the back that were out of frame at the start are only now visible and they don’t have enough time to load so they come out with low resolution. If I’m in play-in-editor and keep the camera in this frame for a while everything eventually loads and looks nice:
I tried to follow your steps, but my videos still look ok. And I can tell by the way the video renders much slower than real-time (and in a stop-and-go fashion) that it’s waiting for tile loads before each frame. I also confirmed with a debug build and some breakpoints that it’s doing what it’s supposed to do.
One problem I did notice is that it doesn’t wait for the initial tileset.json load. So in my first frame, the tileset was missing completely. But once the tileset was available at all, it correctly waited for all tiles before snapping each frame. A warmup should work around that no problem, though.
So, I’m probably doing something a little bit different from you. Would you mind walking me through it in more detail? For example, you say you “created a Cine Camera Actor”, but how did you do it? With the button on the Sequencer panel? That’s what I did. How did you set up the camera flight?
Maybe the easiest thing would be to send me a copy of your Samples project and tell me exactly what buttons to push to reproduce the problem after opening it?
I screen recorded the steps to reproduce the problem and the rendered output is also inside the project folder, in Saved>MovieRenders.
I used spatial samples for this render just to make the issue more noticeable in the render without the additional motion blurring from the temporal samples, the problem is there regardless of what samples are used though.
Thanks for the video. I see the problem, now. It comes down to a difference in how we created the Level Sequence. I did it like this:
But you did it by right-clicking in the Content Browser and adding it there. The critical difference between these two is that my approach added the level sequence to the level, whereas yours just added it as an asset. So then this code in Cesium for Unreal’s Cesium3DTileset.cpp couldn’t find your level sequence:
And so doesn’t know it’s recording a video, and so it doesn’t switch into the “wait for tile loads before snapping the frame” mode.
Easy solution: drag your level sequence from the Content Browser into the viewport, which will add it to the level. Then the recording should behave as intended.
I’m not quite sure what we can do to fix this so that the workaround isn’t needed, but at least the workaround is fairly easy. I wrote an issue to see if we can find a proper fix:
Thanks for your help and patience tracking this down!
I have set up the level sequence from the dropdown menu as described above.
Then I added my camera.
Then I tried to render it using the Movie Scene Capture (legacy)). A window opens with the camera view. But no tiles are loaded yet. Then it hangs here for a minute or so before it actually starts capturing frames.
The first 1-5 frames or so are completely black. Why is that? After that I can see that the tiles are loaded for the next frames.
If I use the (new) Movie Render Queue I have a simular situation. It says “Cut State: Warming Up” for a minute or so. Then it starts rendering the frames the first 1-5 frames just show the sky and black bottom. The frames after that seem to have the tiles.
If I lower the max screen space error then even more frames will be black.
Is it possible to avoid this problem? It should wait for all tiles in the view to finish loading before capturing frames.
Yep I added it from the Movie clip icon at the top. Then add Level sequence as described in your previous reply.
I also tried the other method to drag the level sequence into the viewport.
It might be related to my tile settings:
Maximum Screen Space Error: 15
Preload Ancestors: OFF
Preload Siblings: OFF
Forbid Holes: OFF
Maximum Simulatneous Tile Loads: 100
Loading Descendant Limit: 1000
Enable Frustum Culling: ON
Enable Fog Culling: ON
Enforce Culled Screen Space Error: ON
Culled Screen Space Error: 16
I just want to capture single frames so longer waiting times for a better quality is fine.
Right now the first frame is just black.
Then the next 10 or so frames show the horizon but a black landscape.
A bunch of frames later it will start showing the tiles.
Once it starts showing tiles, does it show all the tiles? Or are there still some missing or an incorrect LOD? There is a problem, as I mentioned above, where the video capture doesn’t wait for the initial load of the tileset’s metadata. Just adding a few seconds of warmup should solve that, though.
Beyond that, please see if you can reproduce this problem in the Cesium for Unreal Samples project, or if it only happens in your own project. Can you give me detailed instructions (or maybe a video showing the steps) to reproduce it?
then it renders the frames just fine with all the tiles
sometimes there is a frame that is only partly rendered. Sometimes the top or bottom is black or just a blue color - looks very glitchy
sometimes one of the following frames is just black
if I increase the resolution to let’s say 4000x4000 px it shows more frames black, some tiles only loaded in very low res, some glitchy half rendered frames, missing tiles
first frame shows the horizon but the ground is completely black
Then all frames are showing the tiles but the tiles at the bottom/top of the image are missing (it is showing the grid below though) - so this is related to culling not working on different aspect ratio in the new MRQ. In the legacy Movie Scene Capture it does not have the culling problem.
The MRQ has no glitches like the legacy one has
if I increase the resolution to 4000x4000 px it shows more frames with just the horizon and black flor. After a few frames it renders fine except for the culling issue.
I don’t have any glitches in MRQ.
So in summary I think the Movie Scene Capture (legacy) is not usable for high res images? Always weird glitching happening.
And the Movie Render Queue (new) has the problem with the first few frames and culling.
The only way I got sort-of-ok results was when I use the MRQ + 5000 warm up frames + culling disabled.
But this results in very long render times and memory usage so some big views I cannot render at all.
However - I downloaded the Cesium example maps again. This time using the newest UE 5.2.1 version.
I used Map12, added LevelSequencer and a Camera.
I manually set the Camera FOV to 90 and Aspect Ratio to 1 in the Camera properties.
Then I rendered the sequence with the MRQ => culling problem appeared again.
Then I added the CameraManager blueprint setup which for now “seems” to work sometimes.
Some views / camera rotations I have no culling issues but in some cases I do have culling issues.
CameraActor => Get Actor Location => Make Cesium Camera (location)
CameraActor => Get Actor Rotation => Make Cesium Camera (rotation)
FieldOfView => 90
AspectRatio => 1,0
[For some reason I cannot do set field of view or set aspect ratio in the blueprint. If I do it doesn’t update the CesiumCamera settings]
For the Tile level of detail set to 16, loading descendant limit to 20 and render resolution at 1000x1000 px it seems to render fine.
HOWEVER - if I set the tile resolution to for example 4, loading descendant limit to 1000 and the render resolution to 4000x4000 px - I get only horizon frames with black bottom.
I guess I can set the warmup frames to 5000-10000 or more but that seems to be very time consuming?
Also I checked with the blueprint to get a notification when the tiles are loaded.
I noticed that it sends the notification multiple times. Even when the tiles are just showing up as grids or only the surface.
I assume the MRQ thinks that the first loaded notification means it should start the render already.
When I click on simulate in the editor and look at the console I can see that it shows something like this:
Im having a similar issue where the cesium tiles in the background of my shot arent being rendered to a high enough LOD for my movie sequence. im rendering at 8k and the low poly look is very noticeable. I have my level sequence in my unreal engine scene, and render warm up set to 800 frames, any ideas on what I can try to fix this?
It’s not obvious from your screenshot what the problem is. But in general, if you want more detail, reduce the Maximum Screen Space Error property. That’s assuming that the movie is capturing the “final” level-of-detail, and the problem is just that that level-of-detail isn’t good enough.
If you think the final LOD isn’t being captured, then decreasing the Max SSE won’t help, and we’ll have to dig into why that’s happening. The most common cause is that the Level Sequence is not in the level (it’s just an asset).
Hey @Kevin_Ring thanks for the message. Ive got Maximum screen space error set to 2, and the level sequence is in the scene too. Im wondering if theirs a way I can increase or make cesium take as much time as it needs to load before it renders a frame?
That should happen automatically, just by using a Level Sequence that is embedded in the level. If you don’t think it’s happening your case, please give us more information. Your screenshot above looks “fully loaded” to me.
Hey @Kevin_Ring it does look fully loaded in that shot. but what happens is along the camera animation sequence in the render, the lods in the distance still “pop” in as the camera moves to them. is this to do with our cesium ion license not being applied to our account?
