It would be great if there’s a way to not load the 3D tiles into memory if they are culled out of range
This happens by default. However, tiles previously loaded are only unloaded after the Maximum Cached Bytes is exceeded. So setting this to a smaller value (even zero is ok!) will reduce memory usage.
There’s an issue with Google Photorealistic 3D Tiles specifically where memory usage increases gradually as you fly around and never goes back down without completely refreshing the tileset. This is the issue, and we expect to address it soon:
A workaround is to periodically call RecreateTileset on the Cesium3DTileset. That will cause all the currently-visible tiles to reload, but memory usage will be reduced.