Memory leak advice

Hi all,

Our configuration is:
Windows 7
Chrome version 49
CesiumJS version 1.19

Our web app consists of many different layers of data to be plotted on a cesium map. We are using the Viewer (var viewer = new Cesium.Viewer). Each layer has its own variable that stores the data source collection, so for layer X we have
var XDataSources = new Cesium.DataSourceCollection();

If a user wants to see data about ABC, we use a REST API and retrieve the data from our backend. Then if the data belongs to layer X, we create a new DataSource (new Cesium.CustomDataSource) say Y, then for each data point retrieved, we add an entity in Y's entity collection.

Finally once all the entities are populated in Y. We add the datasource to our Viewer's datasource collection:

viewer.dataSources.add(Y)

The reason we store all these different datasources is so when the user decide to remove ABC from the map, we keep it in memory, and if he/she decides to add ABC again we don't call the API, we plot it from memory. So the datasources don't get destroyed to avoid repeat API calls.

This has been working great for most of our use cases however, we recently had a query that returned 32K records to be plotted, meaning we created 32K entities. What's happening is that the browser memory ramps up to 1.8Gb and then breaks. In some computers it goes to 1.2GB and plots but still ramps up very quickly. We check the memory using Windows Task Manager.

Any thoughts?

By the way, we have tried the same exercise in IE and Firefox with the same result. Also using the Chrome Dev Tools, it seems like when adding all the entities to the custom data source is when the memory ramps up. I'm not a memory profile expert but that's as close as I got.

Hello,

Thanks for providing all of the details! Cesium should be able to handle creating that many entities, so I’m not really sure where all of the extra memory usage is coming from. It might be related to how you’re adding the entities to the datasource. What type of entities are you creating? Is there a way you could write a code sample to reproduce it with dummy data?

Thanks,

Hannah

Hi Hannah,

I tried to simulate exactly how we are storing the entities and creating them here

http://jsfiddle.net/90nc1u0r/1/

I have made up the entity count to be 3500 in the example but as you can see in task manager the memory just ramps up very quickly.

We are using Dict() has a hashmap as we need to distinguish between datasources. So if user wants to DEF instead of ABC feature points, we can just quickly remove them from the map and yet store them in the client in case they want to see ABC again (so we don't have to make another API call to the backend).

Great, thanks for the code sample! Nothing jumped out at me that would be taking up so much memory.
However, you might benefit using the Primitive API instead of the Entity API since it doesn’t look like the properties of your points are going to be changing after they are created. That change should reduce the memory usage.

Here is a demo of how to add points using a PointPrimitiveCollection: http://cesiumjs.org/cesium/Apps/Sandcastle/index.html?src=development/PointPrimitives.html&label=Development

You can add the PointPrimitiveCollections to your hashmap, then call show/hide on them as needed.

Alternatively, it might be more efficient to store the raw data in your hashmap instead of storing a bunch of DataSources.

Best,

Hannah

Thanks Hannah, will look into PointPrimitiveCollections

I saw the sandcastle example, but how do I show/hide them? If we use PointPrimitiveCollections then we can't be using DataSources right as DataSources only have an EntityCollection.

Does PointPrimitives allow an infobox to pop when you click on them?

No, you would use PointPrimitiveCollections instead of using DataSources. They don’t have InfoBox functionality by default, but it’s pretty easy to add. Below is an example of how to create your own InfoBox. You can toggle show on the points by looping through the collection. I’ve added that to the code example below also.
If you need other features from the Entity API like selecting and tracking an entity, you will have to stick to your current method using DataSources.

var viewer = new Cesium.Viewer(‘cesiumContainer’);
var scene = viewer.scene;

var pointPrimitives = scene.primitives.add(new Cesium.PointPrimitiveCollection());
pointPrimitives.add({
id: {
name: ‘red’,
description: ‘red point’
},
position : Cesium.Cartesian3.fromDegrees(-75.59777, 40.03883),
color : Cesium.Color.RED,
pixelSize : 5
});
pointPrimitives.add({
id: {
name: ‘blue’,
description: ‘blue point’
},
position : Cesium.Cartesian3.fromDegrees(-80.50, 35.14),
color : Cesium.Color.BLUE,
pixelSize : 10
});
pointPrimitives.add({
id: {
name: ‘green’,
description: ‘green point’
},
position : Cesium.Cartesian3.fromDegrees(-80.12, 25.46),
color : Cesium.Color.LIME,
pixelSize : 20
});

var infoBoxContainer = document.createElement(‘div’);
infoBoxContainer.className = ‘cesium-viewer-infoBoxContainer’;
viewer.container.appendChild(infoBoxContainer);
var infoBox = new Cesium.InfoBox(infoBoxContainer);
var infoBoxViewModel = infoBox.viewModel;

var handler = new Cesium.ScreenSpaceEventHandler(scene.canvas);
handler.setInputAction(function(click) {
var pick = scene.pick(click.position);
var showSelection = false;
var titleText = ‘’;
var description = ‘’;
if (Cesium.defined(pick) && Cesium.defined(pick.id)) {
showSelection = true;
titleText = Cesium.defined(pick.id.name) ? pick.id.name : ‘’;
description = Cesium.defined(pick.id.description) ? pick.id.description : ‘’;
}
infoBoxViewModel.showInfo = showSelection;
infoBoxViewModel.titleText = titleText;
infoBoxViewModel.description = description;
}, Cesium.ScreenSpaceEventType.LEFT_CLICK);

var show = true;
Sandcastle.addToolbarButton(‘Toggle Show’, function() {
show = !show;
for (var i = 0; i < pointPrimitives.length; i++) {
pointPrimitives.get(i).show = show;
}
});

``

Best,

Hannah

Thank you very much Hannah, that was very helpful. I'm gonna try that and see how it goes.

Previously since all our entities were batched up per DataSource, it was simple to show/hide based on user's action. I would just hide the whole datasource.

Now I'll have to loop through all points in the PointPrimitiveCollections and determine which ones to hide based on the user input. Is there a way to group the PrimitivePoints within the collections?

You can’t group points within a collection. I think the easiest thing to do would be to make a new collection for each group of points that are in the same dataset.

-Hannah

Thanks Hannah, I ended up doing just that and so far so good.

Question, for performance, is it faster to remove and then add huge amount off primitive points OR to flip the show to true and false?

I chose to remove and add so I don't have to loop through every single point and flip the show flag but not sure if that was the better option.

I think it is better to loop through and change the show value.
When you remove the primitives, it loops through and destroys each point. So either way you, something is looping through the list of points. However, if you show/hide, you don’t have to recreate all of the primitives every time. Creating the primitive objects is the thing that takes the most time.

Best,

Hannah