Shear amount of data processing grinds Cesium framerate to a Hault

1. A concise explanation of the problem you’re experiencing.

I working with Sensor and Positioning Data via Websockets. Data comes in via socket to the browser, and get put into the CZML variables that we have. I then call CzmlDataSource.process(data_variable); to put it on the UI. This works great for like 30 - 50 secs, And then performance of the UI starts to diminish. It slowly over the course of about 60 seconds drops to 1 fps.

I’m processing approximately 2000 packets/minute. I’ve got data that is updating at about 15 hz (fps) and sending directly to the UI via websocket.

Looking at the performance data on the profiler it appears that in the beginning it takes about 3milliseconds to make the DataSource.process() call. After about a 90secs that metric is taking about 30ms.

2. A minimal code example. If you’ve found a bug, this helps us reproduce and repair it.

ws.addEventListener(“message”, function(e) {

var msg = JSON.parse(e.data);

console.log(msg);

if (msg.data.id === “layer1”) {

var timestamp = msg.data.timestamp;

//get availibility string from timestamp

var tempISO = new Date(timestamp).toISOString();

//console.log(msg);

if (msg.data.camera_id === 1) {

//console.log(msg);

//console.log(msg.data.vertices);

//convertheights(msg);

camera_data[11].position.cartographicDegrees.push(tempISO, msg.data.vertices.v0.x, msg.data.vertices.v0.y, msg.data.vertices.v0.z);

camera_data[12].position.cartographicDegrees.push(tempISO, msg.data.vertices.v1.x, msg.data.vertices.v1.y, msg.data.vertices.v1.z);

camera_data[13].position.cartographicDegrees.push(tempISO, msg.data.vertices.v2.x, msg.data.vertices.v2.y, msg.data.vertices.v2.z);

camera_data[14].position.cartographicDegrees.push(tempISO, msg.data.vertices.v3.x, msg.data.vertices.v3.y, msg.data.vertices.v3.z);

camera_data[15].position.cartographicDegrees.push(tempISO, msg.data.vertices.v4.x, msg.data.vertices.v4.y, msg.data.vertices.v4.z);

if (camC.count > 10) {

camera_data[11].position.cartographicDegrees.splice(0, 4);

camera_data[12].position.cartographicDegrees.splice(0, 4);

camera_data[13].position.cartographicDegrees.splice(0, 4);

camera_data[14].position.cartographicDegrees.splice(0, 4);

camera_data[15].position.cartographicDegrees.splice(0, 4);

}

camC.count++;

}

else if (msg.data.camera_id === 2) {

//console.log(msg);

//convertheights(msg);

camera_data[16].position.cartographicDegrees.push(tempISO, msg.data.vertices.v0.x, msg.data.vertices.v0.y, msg.data.vertices.v0.z);

camera_data[17].position.cartographicDegrees.push(tempISO, msg.data.vertices.v1.x, msg.data.vertices.v1.y, msg.data.vertices.v1.z);

camera_data[18].position.cartographicDegrees.push(tempISO, msg.data.vertices.v2.x, msg.data.vertices.v2.y, msg.data.vertices.v2.z);

camera_data[19].position.cartographicDegrees.push(tempISO, msg.data.vertices.v3.x, msg.data.vertices.v3.y, msg.data.vertices.v3.z);

camera_data[20].position.cartographicDegrees.push(tempISO, msg.data.vertices.v4.x, msg.data.vertices.v4.y, msg.data.vertices.v4.z);

if (camC.count > 10) {

camera_data[16].position.cartographicDegrees.splice(0, 4);

camera_data[17].position.cartographicDegrees.splice(0, 4);

camera_data[18].position.cartographicDegrees.splice(0, 4);

camera_data[19].position.cartographicDegrees.splice(0, 4);

camera_data[20].position.cartographicDegrees.splice(0, 4);

}

}

//set clock multiplier

camera_data[0].clock.multiplier = 1;

//process data to the UI

DS_layer1.process(camera_data);

}

3. Context. Why do you need to do this? We might know a better way to accomplish your goal.

How do I make it so the UI stays stable here?

Also a note: I don’t need the data to be persistent at the time of this post.I only need to show a couple seconds worth in either direction.

The speed and granularity of the data is great, if not, exactly what I need in the beginning of the program… but as time goes on well this problem occurs.

4. The Cesium version you’re using, your operating system and browser.

The Cesium Version is 1.41 and is rendered using a Nvidia M2000M GPU. The system has an i7 and 32gb of ram. While its on a laptop, that still pretty good.

Running on Windows 10 gold load, with google chrome as the browser

Hi Alex,

It may make sense to update the entities directly with something like a SampledPositionProperty or CallbackProperty, rather than re-processing the CZML.

You could use the lower-level primitive API directly, which would be more efficient, but you would need to manage updates yourself.

Thanks,

Gabby

Gabby,

Thanks so much for the response, apologies for not seeing the early, I must have missed the notification.

I will keep you updated on the solution path for follow up questions.

Thanks,

Alex

Out of curiosity were you able to solve this issue? I’m seeing something similar using web sockets and czml.

Seems like things run fine for a short time then the browser starts lagging.

Curious why czml is not the preferred option as this seems like an appropriate use-case for the czml data source process method...

My problem hasn’t been solved yet, because I’m a beginner and I don’t know any better way to solve it.

If you have good methods, please advise.

Hi Gabby

I’m seeing a similar issue with the latest release of Cesium (1.53). Why wouldn’t czml be the preferred option as this seems like an appropriate use-case for the czml data source process method…

Thanks,

Jason

It looks like “Develop a system for easy streaming of dynamic data through polling, EventSource, and WebSockets available to all data sources.” is on the roadmap for data sources:

Although there hasn’t been much activity on it for a while. Feel free to bump that. Have you had good results with directly creating the entities yourself?