Handling large feature layers (server+client side question)

CesiumJS
1

Given some large (few to many GB og OGR data sources) i am going to put together a few scripts that will tile up the data source at different zoomlvls. At each zoom lvl simplication and feature removable will be done to ensure that tiles sizes are keept down by not showing geometries that would be to small anyway to be seen.

I have been using ol3 mostly up until now and have been using geojson and topojson format for my tiles.

Have anyone on the cesium dev team been giving vector tiles any thoughts that they can share. Would the advice be to generate output in czml instead?

Are there any providers in cesium right now that would be able to handle vector layers at differnet zoom lvls, such the scene would be regenerated with new featuers when the view changes resolution.

I am also open to other suggestions. Only goal here is to be able to render and visualize large vector layers from pregenerated tiles to make data more accesiable and not rely on postgis/ wms/wfs servers.

2

Hi Poul,

We have plans to support vector tiles: #2132. Details are still TBA, but this will be an optimized, tiled, binary format.

In the meantime, it would be reasonable to stream GeoJSON/CZML/etc., but you will have to handle cracks at the tile boundaries where two different LOD levels meet - this is a problem that is unique to 3D.

Although it is not part of the public API, check out QuadtreePrimitive.

Patrick

3

Cool.

I have been spending a fair amount of time getting to know all the alternatives out there for vector tiles and will be looking very much forward to see what you guys comes up with. I will help out in any way I can. I will share some of the conclusions I have generated in this process. My background is mostly in software engineering and less in GIS, so use this as you see fit, i most likely wont know everything on this topic and you guys properly already spend som time thinking about it.

The thing I hate the most with the current vector tiles implementations is that it renders each lvl of detail as separate layers. I see the advantage ofcause, that it can be handled at tile generation time what features are shown at different lvls. But it also means that the client is rendering a vector layer for each lvl of detail.

I dont know if it will be possible, but instead of this vector layer per lvl of detail I have come to the conclusion that if I was designing it, I would use the resolution at the given camera position to determine which featuers are returned from the server.

This means that in case the view start at zoom lvl N, the client would have to ask for the tiles in lvl og detail N,N-1,…0 to get all features for the current view. Tile N could of cause have a byte telling how many lvls of details it would need to traverse back to get all features or simply a flag telling if it has any parent features.

This means that if starting at LOD 0 and starting zooming in, the client already have the large covering features and as zooming in, the new tiles only bring new features that have not been presented from parent tiles.

One could discuss if the server should do simplification on features. Its a valid point, but then when applying above, I think the client should be able to just update the already rendered feature if a less simplified feature is returned in a child tile.

On tile boundaries, I would apply similar as the quantized-mesh by clipping on tile boundaries and the client can be responsible of combining features with same id again. Polygons becomes linestrings ect, where there will be a point on the tile boundary if it extent one tile. But if we apply a technique as above, there would never be any need to clip features, because they can always be returned in the parent tile that has extent to contain the hole feature.

Having worked with geojson tiles, what seems to bring up the tile size is the amount of properties that we put into features these days. I believe that geometry and properties should be separated such the client can stream very light weight tiles for drawing the geometries, and then on demand or in separate tiles fetch properties since in many cases you really dont click or use this information. Also applying above technique with fetching all parent tiles, no features are split over multiply tiles and it is then possible to get the properties also for the same tiles without having to decide which tiles the properties goes in.

I am very interested in moving vector tiles forward and as soon as we can have a spec for these tiles, I can contribute with some backend stuff for the .net/mssql community (thats our area of focus). I will keep an eye out for the issue you mentioned.

4

Thanks for your enthusiasm.

I would use the resolution at the given camera position to determine which featuers are returned from the server.

Yes, this is exactly how it will work similar to how we stream image tiles now.

I believe that geometry and properties should be separated such the client can stream very light weight tiles for drawing the geometries, and then on demand or in separate tiles fetch properties since in many cases you really dont click or use this information

Yes, for sure!

Keep an eye on #2132 for updates on this. In the meantime, watch #2179 for the low-level graphics work we are doing now that we will build vector tiles on.

Patrick

5

I also spend half a day playing more in depth with how mapbox has defined vector tiles. I just dumped my stuff on github. There is also some javascript for decoding them.

I really think its amazing how compact the pbf format becomes. i have some tiles (46KB) that is 20x smaller than the similar geojson(800KB) file and even 3 times smaller than gziped geojson (128kb).

What I miss the most is a better handling of the stuff that happens at tile borders. Clipping is easy, but there seem to be missing som focus on the client side for combing them again.

Yes, this is exactly how it will work similar to how we stream image tiles now.

But image tiles are more trivial, you always want to show the better quality image tiles as zooming in. With vectors we kinda have the benefit that we can return full resolution geometries many zoom lvls before its needed. With ol3 where i played with vector tiles, I dont like it 100% how when going from one zoom lvl to another that it rerenders the hole vector layer. Even with simplified features in parent tiles, it seems like alot of redundency to return the same features for each zoom lvl.

Maybe what I am missing is more tooling/controll for the generation of vector tiles in what happens at each zoom lvl. As an example in mapbox tilelive bridge https://github.com/mapbox/tilelive-bridge/blob/master/index.js#L154 its coded to simplify with a tolerance of 8 for all lvls except of the last layer. Maybe 8 is the magic number :slight_smile:

But great work guys. Will be fun to see what comes.

6

I can’t commit yet, but it’s likely that each higher-resolution LOD will add vertices to the previous LOD, so that all the nodes in the tree is O(n) size compared to the input data, as opposed to nlog(n). For example, if an input polygon had 84 vertices, the root node might have 4 vertices, then the children have 4 each (16 total), then those children 4 each (64 total). Each node would be a subset of the actual polygon vertices, not an approximation. This is similar to the Layered Point Cloud approach:

http://www.crs4.it/vic/data/papers/spbg04-lpc.pdf

Patrick

7

Then I would be happy atleast :slight_smile:

8

What I miss the most is a better handling of the stuff that happens at tile borders. Clipping is easy, but there seem to be missing som focus on the client side for combing them again.

Recombining geometries may not be necessary. With buffered vector tiles and using some clipping at rendering time it should be possible to eliminate rendering issues at the tile boundaries.

But image tiles are more trivial, you always want to show the better quality image tiles as zooming in. With vectors we kinda have the benefit that we can return full resolution geometries many zoom lvls before its needed. With ol3 where i played with vector tiles, I dont like it 100% how when going from one zoom lvl to another that it rerenders the hole vector layer. Even with simplified features in parent tiles, it seems like alot of redundency to return the same features for each zoom lvl.

Right, the "vector tile" implementation in ol3 is… prototypish. I hope we can improve it in the future. In particular, we should reuse "parent" tiles while waiting for "child" tiles.

Cheers,