Best approach to process lots of real time data

Hi all,

I’m currently working on a project where I receive lots of sensor data on my server, which I pass to the frontend using WebSockets to visualize it.

I do have 5 sensors which are visualized by 5 spheres (EllipsisGraphics in an EntityCollection). These spheres have changing radii and colors, depending on the sensor value.

Now these values fly in with about 1-5 per second. What’s the best way to process this in the frontend? My current set up (Geforce GTX 960 with 3.4 GHz octa core) is slow at rendering - new data arrives faster than my PC can draw (about 1-2 per second, instead of 1-5). Also, I’m using some transparency and the spheres flicker when being updated - it seems like they have no transparency when rendering first, then the transparency comes back. Removing transparency doesn’t make it faster though.

I thought about the following approaches:

  1. cache incoming data at the frontend for 1s, suspend+resume EntityCollection events and only force rendering once a second. This is slow when using 10 sensors (spheres) and also on slowlier devices. Is there a more generic method possible?
  2. the primitives API seems to be a bit more performant (btw, is there some guide about the differences?). While I can update the color nicely, how can I update the radius?
  3. shall I collect the data at the server and push it out using a CZML stream via Websocket? Does this help performance?
  4. is it possible to cancel the current rendering process?
    As it’s only some spheres with changing radii and colors which should be drawn at quite some FPS, I wonder why it’s thaaat slow. Is there something that I overlooked? Are the ideas going into the right direction? Shall I rather post my current code so we can find the reasons why it’s so slowly?

Thanks in advance

Sebastian

I have similar setup, with around 20,000 entities and around 50-100 updates/second via web socket. While there is some performance issue, as long as there are up to 20,000 entities it runs pretty well on strong machines (on older hardware the limit varies between 2000 to 10000 entities).

I’m working on some more improvements to the rendering cycle, which hopefully would lead to better results. I also have some issues with the KML dataSource, which seems to cause a performance issue of its own.

Actually, with data that changes constantly, the entities show surprisingly good results - sometimes better than using primitives.

Regarding your issue:

Can you post your entities.add code?
Have you tried using CallbackProperty for the dynamic values?

On second thought - post your whole code. The issue might be at some other point in the code.

Hi Yonatan,

thanks for your encouraging post! Knowing that it should be possible, I looked over my code and could narrow down the problem. I wasn’t using material proprties for my radii and materials and thus the redrawing was quite slow.

Here’s some code that can be run in the sandbox, which shows which way I’m currently going in my application. Is there still something that can be improved?

var config = {

initialView: {

center : Cesium.Cartesian3.fromDegrees(13.723059, 51.025410),

offset : new Cesium.HeadingPitchRange(

0,

Cesium.Math.toRadians(-45),

500

)

}

};

var viewer = new Cesium.Viewer(‘cesiumContainer’);

viewer.scene.skyAtmosphere.show = false;

viewer.camera.flyToBoundingSphere(

new Cesium.BoundingSphere(config.initialView.center), {

offset: config.initialView.offset,

duration: 0

}

);

var sensors = [

[51.025343,13.723303, 10],

[51.025289, 13.722950, 3],

[51.025695, 13.722740, 20],

[51.025748, 13.722368, 40]

];

var getEllipsoid = function(entityIndex) {

return {

radii : new Cesium.CallbackProperty(function() {

return new Cesium.Cartesian3(entities[entityIndex].value, entities[entityIndex].value, entities[entityIndex].value);

}, false),

material : new Cesium.ColorMaterialProperty(new Cesium.CallbackProperty(function() {

return entities[entityIndex].value < 10 ? Cesium.Color.GREEN.withAlpha(0.5) : Cesium.Color.RED.withAlpha(0.5);

}, false))

};

};

var entities = ;

var defaultValue = 10;

for(var i = 0; i < sensors.length; i++) {

entities[i] = viewer.entities.add({

name: 'Sensor ’ + i,

position: new Cesium.Cartesian3.fromDegrees(sensors[i][1], sensors[i][0], sensors[i][2]),

value: defaultValue

});

entities[i].ellipsoid = getEllipsoid(i);

}

var getRandomInt = function(min, max) {

return Math.floor(Math.random() * (max - min + 1)) + min;

};

var intervalId;

var startAnimating = function() {

stopAnimating();

intervalId = setInterval(function() {

var randomIndex = getRandomInt(0, entities.length - 1);

entities[randomIndex].value = getRandomInt(1,20);

}, 200);

};

var stopAnimating = function() {

if(typeof(intervalId) !== “undefined”) {

clearInterval(intervalId);

intervalId = undefined;

}

};

var isPlaying;

viewer.clock.onTick.addEventListener(function(clock) {

var newIsPlaying = clock.shouldAnimate;

if(newIsPlaying && !isPlaying) {

startAnimating();

}else if(!newIsPlaying && isPlaying) {

stopAnimating();

}

isPlaying = newIsPlaying;

});

``

Thanks in advance

Sebastian