Fps drop while rendering sparse point cloud data as 3D tiles

Summary

Hello,

I am using py3dtiles to convert point clouds to cesium 3d tiles. I am facing issues when trying to load sparse point cloud datasets (Terrain) on cesium viewer, there is significant drop in frame rate while trying to load all the points in the field of view of the camera which lags the whole system.

I don’t see any bottleneck in compute resource as RAM is used upto 60% and CPU utilization is barely 40%.

Can anyone help me on debugging this issue? Thanks in advance.

Steps to reproduce

  1. I am converting this file using py3dtiles. The resultant tileset consists of 4755 files and total size of 381.5 MB Here is the output tileset from py3dtiles.
  2. For loading tileset, const tileset = await Cesium.Cesium3DTileset.fromUrl('output/tileset.json'); on cesiumJs version 1.111.0.

Informations

  • OS: Ubuntu 22.04
  • CesiumJs: 1.111.0
  • Google chrome: 120.0.6099.71
  • Hardware: Lenovo thinkpad E14.

There could be different reasons for why you see an unexpectedly low performance here.

There are some aspects that might be related to the data set and the structure of the tileset itself.

The data set contains 25 million points. Displaying these all at once may impose a considerable workload for any “standard PC”. I tried it out locally, and when ~21 million points of ~4000 tiles are displayed on my deskop PC, it still had >30FPS, but on smaller laptops, this may just be too much…

One option that you could try: When loading the tileset, you could set a higher value for the maximumScreenSpaceError. The default value is 16, and when you set

const tileset = await Cesium.Cesium3DTileset.fromUrl(
  "http://localhost:8003/tileset.json", {
   maximumScreenSpaceError : 32
});

then it should try to load fewer tiles. This means that you won’t see all points at once, but maybe it increases the FPS to an acceptable level.

The points of that LAZ file are split into ~4750 files, and most of them are very small. This means that the viewer has to traverse a relatively large tile hierarchy, where each tile only contains ~“relatively little” data. I had a short look at py3dtiles to see whether it offers some configurability here, but didn’t see anything obvious in the CLI documentation - maybe, some code could be tweaked internally.

You could also consider to use https://ion.cesium.com/ instead of the py3dtiles library. The tutorial at Point Clouds – Cesium describes how you can tile LAZ files, and it might be that you can achieve a better performance with the resulting tileset (unless the limit really is the GPU of your laptop which may be struggling with trying to display 25 million points…).

Thank you for your response and detailed explaination,

I undertand displaying 25 million points on a smaller laptop could be a considerable workload on the machine. But as per my observation I could not find any considerable workload on my machine while the points were being displayed.

The FPS was around 1-4 frames. The whole system was laggy but this is what I could observe in my system monitor

I also tried to increase the maximumScreenSpaceError to 32 as you suggested I could not see any improvement in performance.

Please let me know if I am doing anyting wrong, here is the contents of index.html I am using liveserver vs code extension to load on the browser.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="utf-8">
    <script src="cesium-main/Build/CesiumUnminified/Cesium.js"></script>
    <style>
        @import url(cesium-main/Build/CesiumUnminified/Widgets/widgets.css);
  
        #cesiumContainer {
            position: absolute;
            top: 0;
            left: 0;
            height: 100%;
            width: 100%;
            margin: 0;
            overflow: hidden;
            padding: 0;
            font-family: sans-serif;
        }
  
        html {
          height: 100%;
        }
  
        body {
            padding: 0;
            margin: 0;
            overflow: hidden;
            height: 100%;
        }
    </style>
</head>

<body>
    <div id="cesiumContainer"></div>
    <script type="module">


        const viewer = new Cesium.Viewer('cesiumContainer', {
            terrain: Cesium.Terrain.fromWorldTerrain(),
        });

        const position = Cesium.Cartesian3.fromDegrees(
            76.61758113143246,
            28.192182356673207,
            1
        );
        const orientation = Cesium.Transforms.headingPitchRollQuaternion(
        position,
        new Cesium.HeadingPitchRoll.fromDegrees(0, 0, 0)
        );
        const scaling = new Cesium.Cartesian3(1, 1, 1);

        const modelMatrix = Cesium.Matrix4.fromTranslationQuaternionRotationScale(
            position,
            orientation,
            scaling
        );
        
        const tileset = await Cesium.Cesium3DTileset.fromUrl('3dtiles/bc_3_1_4/tileset.json',{
            maximumScreenSpaceError : 32,
        });
        tileset.style = new Cesium.Cesium3DTileStyle({
            pointSize : 2.0,
        }
        
        )
       

        viewer.scene.primitives.add(tileset);
        tileset.root.transform = Cesium.Matrix4.IDENTITY;
        tileset.modelMatrix = modelMatrix;
        viewer.scene.debugShowFramesPerSecond =true;

        viewer.terrainProvider = new Cesium.EllipsoidTerrainProvider();

        viewer.flyTo(tileset);
    </script>
    </div>
</body>

</html>

Your CPU might be bored. But your GPU might be (too) busy.

When running the code that you provided in a local sandcastle, then this is the result for me:

The CPU is nearly idle at ~9%. But the GPU is busy, 100%, with “3D Stuff”. (That’s on a Desktop - usually, Laptops have less GPU power, give and take the fact that my GPU is 3 years old and someone might have a brutally optimized gaming Laptop etc)

The hint about the maximumScreenSpaceError may have to be elaborated:

You said that 32 did not have any effect for you. And that’s true: I checked that with 32, it still loads all tiles at once when you zoom out. When you run that code in a sandcastle and insert

viewer.extend(Cesium.viewerCesium3DTilesInspectorMixin);
const inspectorViewModel = viewer.cesium3DTilesInspector.viewModel;

after creating the viewer, you can play around with some settings and gather information about the workload. For example, with 32, it is still loading all 25 million points, and runs at ~14FPS for me (on a Desktop PC at 100% GPU workload). When increasing the maximumScreenSpaceError to 128, it has to display “only” 13 million points, and then it runs at ~30FPS:

Cesium Points Performance

So you may try to increase the maximumScreenSpaceError to a higher value, like 128 or 256 or even more (depending on the trade-off between “number of points” and “FPS” for your system).

Thank you for your response,

I did check my GPU usage indeed it was too occupied. I will consider your suggestion and try to tweak a bit on py3dtilers, to generate less files and I will try to optimize it.

I also tried increasing the maximumScreenSpaceError to 512 and 1024. I could get better FPS. Visually the tileset was looking the same as the original point cloud data, anyways the number of points was significantly reduced which is completely fine.

Is there any way to dynamically change the maximumScreenSpace error based on the FPS on the client machine in runtime? I did check out the dynamicScreenSpaceError property of the tileset but I could not see any significant change in performance unlike maximumScreenSpaceError.

I’m not aware of any “built-in” way of adjusting the maximum screen space error based on the FPS. One could think about some options here.

An overly naive and very basic, pragmatic approach would be to ajdust the MSSE in each frame, with something along the (pseudocode!) line of tileset.maximumScreenSpaceError += (targetFps - currentFps), but

  1. there are some “obvious” things to keep in mind (e.g. some range checks, and the fact that it should probably use something like “the average FPS from the past 10 frames” (instead of the currentFps))
  2. it’s hard to foresee how well this behaves in practice

In fact, it could be an interesting experiment. I’ll consider to play with that (maybe during the weekend), but cannot promise anything right now. (More specific discussions could go into the CesiumJS section of the forum, by the way).

Thank you for your response and insights,

you can let me know if you find anything related to this experiment. Until then I will experiment from my end.

I did a very basic experiment. And I cannot add enough disclaimers here: This is literally just written down within a few minutes, just to see whether it could work in general.

It assumes that the tileset (i.e. your tileset that you shared in the first post) is provided via a server on localhost. The basic idea is: There is an FpsTracker that provides the FPS (averaged over 10 frames. There is an MsseUpdater that regularly updates the maximumScreenSpaceError. It does so by checking the average FPS (every second), comparing it to a targetFps (that is currently set to 50), and based on the deviation from this target, either increases or decreases the MSSE with a certain rate.

From a very quick test, it seems like this could work: When loading your tileset, it starts with the MSSE of 32. Note that initially, the numbers are “distorted” by the fact that it has to load even the initial set of tiles first. But one can see that after a while, it somewhat converges towards a state where it has somewhat stable 50FPS, with an MSSE of ~300:

FPS 36.45643456060094 target 50 change from 32 to 33.6
FPS 41.356492969268764 target 50 change from 33.6 to 35.28
FPS 39.66679888938825 target 50 change from 35.28 to 37.044000000000004
FPS 45.45454545454545 target 50 change from 37.044000000000004 to 38.89620000000001
FPS 46.16805170829732 target 50 change from 38.89620000000001 to 40.84101000000001
FPS 40.338846309238036 target 50 change from 40.84101000000001 to 42.88306050000001
...
FPS 50.81300812988893 target 50 change from 325.4261684785942 to 309.1548600546645
FPS 43.66812227074236 target 50 change from 309.1548600546645 to 324.61260305739773
FPS 50.377833753148614 target 50 change from 324.61260305739773 to 308.38197290452786
FPS 43.290043290043286 target 50 change from 308.38197290452786 to 323.8010715497543
FPS 49.407114624414994 target 50 change from 323.8010715497543 to 339.991125127242
FPS 58.445353594516504 target 50 change from 339.991125127242 to 322.9915688708799
FPS 49.55401387494093 target 50 change from 322.9915688708799 to 339.1411473144239
FPS 58.377116170207266 target 50 change from 339.1411473144239 to 322.1840899487027

There are many degrees of freedom and steering factors. And there certainly will be corner cases. (For example: There might be a tileset where it achieves 60FPS with an MSSE of 100, but only 10FPS with an MSSE of 101 - so there may be no point that it can “converge” to). But the basic approach could be worth another look…

const viewer = new Cesium.Viewer("cesiumContainer", {
  terrain: Cesium.Terrain.fromWorldTerrain(),
});

viewer.extend(Cesium.viewerCesium3DTilesInspectorMixin);
const inspectorViewModel = viewer.cesium3DTilesInspector.viewModel;

const position = Cesium.Cartesian3.fromDegrees(
  76.61758113143246,
  28.192182356673207,
  1
);
const orientation = Cesium.Transforms.headingPitchRollQuaternion(
  position,
  new Cesium.HeadingPitchRoll.fromDegrees(0, 0, 0)
);
const scaling = new Cesium.Cartesian3(1, 1, 1);

const modelMatrix = Cesium.Matrix4.fromTranslationQuaternionRotationScale(
  position,
  orientation,
  scaling
);

const tileset = await Cesium.Cesium3DTileset.fromUrl(
  "http://localhost:8003/tileset.json",
  {
    maximumScreenSpaceError: 32,
  }
);
tileset.style = new Cesium.Cesium3DTileStyle({
  pointSize: 2.0,
});

viewer.scene.primitives.add(tileset);
tileset.root.transform = Cesium.Matrix4.IDENTITY;
tileset.modelMatrix = modelMatrix;
viewer.scene.debugShowFramesPerSecond = true;

viewer.terrainProvider = new Cesium.EllipsoidTerrainProvider();
viewer.flyTo(tileset);

//============================================================================

class FpsTracker {
  constructor() {
    this.previousTimeStampMs = undefined;
    this.frameTimesMs = [];
    this.windowSize = 10;
    this.averageFps = 1;
  }
  update() {
    const currentTimeStampMs = Cesium.getTimestamp();
    if (this.previousTimeStampMs === undefined) {
      this.previousTimeStampMs = currentTimeStampMs;
      return;
    }
    const previousFrameTimeMs = currentTimeStampMs - this.previousTimeStampMs;
    this.previousTimeStampMs = currentTimeStampMs;
    this.frameTimesMs.push(previousFrameTimeMs);
    while (this.frameTimesMs.length > this.windowSize) {
      this.frameTimesMs.shift();
    }
    const frameTimesMsSum = this.frameTimesMs.reduce(function (a, b) {
      return a + b;
    }, 0);
    const averageFrameTimeMs = frameTimesMsSum / this.frameTimesMs.length;
    this.averageFps = 1000.0 / averageFrameTimeMs;
    //console.log("Average FPS "+this.averageFps+" from "+this.frameTimesMs);
  }
  getAverageFps() {
    return this.averageFps;
  }
}

const fpsTracker = new FpsTracker();

class MsseUpdater {
  constructor(fpsTracker, tileset) {
    this.fpsTracker = fpsTracker;
    this.tileset = tileset;
    this.previousTimeStampMs = undefined;
    this.updateIntervalMs = 1000;
    this.targetFps = 50;
    this.adjustmentRate = 0.05;
  }

  update() {
    this.fpsTracker.update();
    const currentTimeStampMs = Cesium.getTimestamp();
    if (this.previousTimeStampMs === undefined) {
      this.previousTimeStampMs = currentTimeStampMs;
      return;
    }
    if (currentTimeStampMs - this.previousTimeStampMs < this.updateIntervalMs) {
      return;
    }
    this.previousTimeStampMs = currentTimeStampMs;

    const averageFps = this.fpsTracker.getAverageFps();
    const deviation = averageFps - this.targetFps;
    let newMaximumScreenSpaceError = this.tileset.maximumScreenSpaceError;
    if (deviation > 0) {
      newMaximumScreenSpaceError *= 1.0 - this.adjustmentRate;
    } else if (deviation < 0) {
      newMaximumScreenSpaceError *= 1.0 + this.adjustmentRate;
    }
    console.log(
      "FPS " +
        averageFps +
        " target " +
        this.targetFps +
        " change from " +
        this.tileset.maximumScreenSpaceError +
        " to " +
        newMaximumScreenSpaceError
    );
    this.tileset.maximumScreenSpaceError = newMaximumScreenSpaceError;
  }
}

const msseUpdater = new MsseUpdater(fpsTracker, tileset);
viewer.scene.preUpdate.addEventListener(() => msseUpdater.update());

1 Like

Thank you for your response.

The provided code snippet solved all my performance issues, I made few modifications by providing 2 thresholds, an upper threshold FPS and a lower threshold FPS. as I thought changing the MSSE constantly would affect the user experience,

However, sharing the changes made. Let me know if any improvements can be done. Thanks again for the insights and thoughts on the issue.

//============================================================================

class FpsTracker {
  constructor() {
    this.previousTimeStampMs = undefined;
    this.frameTimesMs = [];
    this.windowSize = 10;
    this.averageFps = 1;
  }
  update() {
    const currentTimeStampMs = Cesium.getTimestamp();
    if (this.previousTimeStampMs === undefined) {
      this.previousTimeStampMs = currentTimeStampMs;
      return;
    }
    const previousFrameTimeMs = currentTimeStampMs - this.previousTimeStampMs;
    this.previousTimeStampMs = currentTimeStampMs;
    this.frameTimesMs.push(previousFrameTimeMs);
    while (this.frameTimesMs.length > this.windowSize) {
      this.frameTimesMs.shift();
    }
    const frameTimesMsSum = this.frameTimesMs.reduce(function (a, b) {
      return a + b;
    }, 0);
    const averageFrameTimeMs = frameTimesMsSum / this.frameTimesMs.length;
    this.averageFps = 1000.0 / averageFrameTimeMs;
    //console.log("Average FPS "+this.averageFps+" from "+this.frameTimesMs);
  }
  getAverageFps() {
    return this.averageFps;
  }
}

const fpsTracker = new FpsTracker();

class MsseUpdater {
  constructor(fpsTracker, tileset) {
    this.fpsTracker = fpsTracker;
    this.tileset = tileset;
    this.previousTimeStampMs = undefined;
    this.updateIntervalMs = 50;
    this.upperFpsThreshold = 40;
    this.lowerFpsThreshold = 20;
    this.adjustmentRate = 0.05;
  }

  update() {
    this.fpsTracker.update();
    const currentTimeStampMs = Cesium.getTimestamp();
    if (this.previousTimeStampMs === undefined) {
      this.previousTimeStampMs = currentTimeStampMs;
      return;
    }
    if (currentTimeStampMs - this.previousTimeStampMs < this.updateIntervalMs) {
      return;
    }
    this.previousTimeStampMs = currentTimeStampMs;

    const averageFps = this.fpsTracker.getAverageFps();
    let newMaximumScreenSpaceError = this.tileset.maximumScreenSpaceError;
    if (averageFps > this.lowerFpsThreshold && averageFps < this.upperFpsThreshold){
        return;
    }
    else if(averageFps > this.upperFpsThreshold){
      newMaximumScreenSpaceError *= 1.0 - this.adjustmentRate;  
    } else if (averageFps < this.lowerFpsThreshold) {
      newMaximumScreenSpaceError *= 1.0 + this.adjustmentRate;
    }
    console.log(
      "FPS " +
        averageFps +
        " target " +
        this.targetFps +
        " change from " +
        this.tileset.maximumScreenSpaceError +
        " to " +
        newMaximumScreenSpaceError +
        " timestamp " + 
        currentTimeStampMs
    );
    this.tileset.maximumScreenSpaceError = newMaximumScreenSpaceError;
  }
}

const msseUpdater = new MsseUpdater(fpsTracker, tileset);
viewer.scene.preUpdate.addEventListener(() => msseUpdater.update());

Yes, the code was really only a first shot, and would require some polishing. Everything that is currently hard-wired (window size, target FPS, adjustment rate and interval, etc) would have to be carefully reviewed, raising the questions about the exact values and/or whether these parameters should be exposed so that they can be tweaked by callers/users.

You also noticed one problem (and tried to solve it): The MSSE ~“converges” to a certain range, but it still fluctuates constantly. The upper/lower FPS thresholds that you introduced are one option for addressing this. And they are probably a good choice insofar that they are intuitive. Maybe the fluctuation of the FPS itself could be reduced with a larger windowSize.

I also thought about making the adjustment to the MSSE depend on the deviation from the target FPS - basically as in

const deviation = (currentFps - targetFps);
const percent = deviation / (targetFps / 100);
newMsse = oldMsse - (percent * (oldMsse/100)); // Large deviation -> large change

But the relationship between the MSSE and the FPS is certainly not linear, so I thought that the “purely iterative” adjustment (with a fixed adjustmentRate) could be more “stable” and help to actually converge to a certain value. (I also noticed that you decreased the updateIntervalMs significantly - another tweaking parameter…)

There are many other things to evaluate. For example: It might also be that there are other elements in the viewer (that are not related to the tileset itself), and when these elements might decrease the FPS, and the MsseUpdater might helplessly increase the MSSE without actually achieving anything.

But when you say that it solved your performance issues in the current form, it’s probably worth investigating this further.

Thank you for the response, and my apologies for late reply.

I will keep your suggestions in mind and use this carefully.