[cesium-dev] KML support

Hello,

I was very happy when my project was accepted into Google Summer of Code 2013, it’s an huge opportunity for me and I will do my best to get the most out of this new experience. I found Cesium by chance but it quickly became my first pick for GSoC, you have a great adventure here and it makes me very proud to be able to contribute to the development of Cesium with a project like this. I hope to learn a lot from all of you!

Pleasantries aside, let’s get to work…
I’ve opened an issue on GitHub called KML Support Roadmap. Matt Amato has been very helpful and composed a draft, I’ve been adding stuff to it so please take a look and comment it.

I would really appreciate feedback in order to better align my project with Cesium’s future goals, after all this is a project that might take longer than 3 months to complete. I’m really excited to start working on this project.

Let’s do this!

André

I have a sort-of-working LineString implementation, if you’re
interested…
it’s based on the ‘batching’ branch and the WallGeometry object in
that branch

Segunda-feira, 17 de Junho de 2013 22:21:50 UTC+1, Ákos Maróy escreveu:

As an FYI, I’m the official GSoC mentor for this task, but I certainly expect most discussion about the work to take place out in the open on this list and the Github issue.

To clarify the task for everyone, Andre is working on high-level KML support, meaning that he will be parsing KML into objects at the DynamicScene level. This is where all of the constructs needed to support KML will live, since it’s already where CZML (and eventually GeoJson) also lives. The actual Scene/Primitive work being done in the batching branch is definitely needed for KML support, but processing the data into DynamicObjects and then rendering those objects are two different things. The in-memory representation of CZML/KML/GeoJson will all share the same architecture.

As for here to start, I would concentrate simply on loading the kml document and iterating over all of the elements in it. Then for each main type (Placemark/Overlay/etc…) we would create a separate function to process it. That function could then further delegate to more specific functions (for example, handling the different geometry types.

I wouldn’t worry about creating a big hierarchy of classes yet, or where the code actually goes. Just do everything in KmlDataSource for now and when we have a better view of how things are shaping up, we can figure out how to break that up into separate source files.

please find my code snippet at the end.
as you see it uses XPath statements to go through a KML file and
create WallGeometry objects from it. this code has been verified in
the batching branch to display KML files like this one:
please note that the namespacing / package is ‘OamViewer’, as this
code was created for the Open Aviation Map project, with the
intention to share it with the wider community
// default KML namespace resolver, see
//
function kmlNsResolver(prefix) {
return ‘’;
}
/**
* Create a set of Walls from a KML document that includes
LineString elements.

Andre / Matt -

Here’s a few outreach efforts to consider for the KML roadmap:

  • Cesium vs. Google Earth performance comparison. As we finish the batching work under-the-hood, I expect Cesium will be faster than Google Earth despite being implemented in JavaScript.
  • Cesium for Google Earth Developers tutorial. This would be a bit broader than KML, but generating and visualizing KML would be the focus.
  • Use Cesium to visualize KML files on GitHub just like Leaflet is used for GeoJSON. Got time on your hands? Use Cesium to edit them too!

Patrick

That’s exciting, it’s nice to see GitHub supporting geographic data directly, hopefully it will attract more talented developers into the area.

They sound like good long term goals, there’s still a long way to go, let’s see what Matt has to say.

Segunda-feira, 1 de Julho de 2013 20:37:36 UTC+1, Patrick Cozzi escreveu:

what I'd really like to see would be examples like
on http://thematicmapping.org/

prism maps


and 3d bars:

This really showcases the ability of a webglobe over a flat map.

Hi,

I started working on a project that aims to build a browser based flight route viewer / player able to work with flight data files recorded in KML format. The player will be integrated into Diamond Aviators site ( www.diamondaviators.net ).

In the next few weeks I will focus extending and fixing KML importer module and add some camera support for Cesium.

Any suggestions are welcome.

Gábor

Gábor,

See our KML roadmap here https://github.com/AnalyticalGraphicsInc/cesium/issues/873 and Andre’s KML branch (where the work is being done) here: https://github.com/andre-nunes/cesium/tree/kml

You should be able to just start out by using the official support Andre is building (via KmlDataSource). If you run into problems or there’s part of the spec you need that’s not implemented yet, we’d be happy to have you work on it with us.

Matt

Matthew,

After spending some days getting familiarized with the code the time arrived to ask for your kind help.

It seems that the way KML styles and objects are merged and eventually transformed to dynamic scene objects has an inherent flaw. I'm still learning the code but to my best understanding all collected styles are mixed together to form a new dynamic object even a particular definition is not applicable to a KML geometry. This leads to geometry misinterpretation.

Given the following KML example

<kml xmlns="http://www.opengis.net/kml/2.2">
  <Document>
    <Style id="flightpath">
      <LineStyle><color>7f00ffff</color><width>4</width</LineStyle>
      <PolyStyle><color>7f00ff00</color></PolyStyle>
    </Style>
    <Placemark>
      <styleUrl>#flightpath</styleUrl>
      <LineString>
        <coordinates>...</coordinates>
      </LineString>
   </Placemark>..

The renderer ends up calculating overlaps and intersections of a polygon consisting of 1500 vertices instead of drawing a simple set of lines just because of existence of .polygon property derived from PolyStyle definition.

I'm working out a fix that applies only certain dynamic properties to the target objet depending on the geometry type of the original KML placemark. Please confirm that I'm going the right direction.
Thanks,

Gábor

2013. augusztus 22., csütörtök 14:33:39 UTC+2 időpontban Matthew Amato a következőt írta:

Thanks for the bug report; I did a quick check of the code and you’re correct. Your fix sounds correct as well. This was just an oversight on our part, since KML support is still in pretty heavy development we simply haven’t tried those kinds of files yet. A pull request into Andre’s branch where the work is being done is most welcome: https://github.com/andre-nunes/cesium/tree/kml

In order to merge your code, we will need you to sign the CLA (http://www.agi.com/products/licensing-and-evaluation-options/individual-cla-agi-v1.0.txt) and email it to cla@agi.com.

Full details in regards to our contribution policies can be found here: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/CONTRIBUTING.md

Thanks

Hi Gábor,

Thanks for reporting that bug, your bug fix would be very appreciated! Feel free to open a pull request on my KML branch. I'll be away most of the week but I'll look at your code as soon as I get back.

Best regards,
André

Dear Matthew,

Thanks for the confirmation. The fix will be shortly available in my branch.

André, please have a look on my branch and review the changes I made before I propose them.

You can find it here https://github.com/segabor/cesium

The CLA will be sent soon.

Regards,

Gábor

Hi Gabor,

We already have a corporate CLA from EU Edge that Akos sent in (see CONTRIBUTORS.md) so we don’t need a separate CLA from you unless you are doing this as an individual, i.e., not at work.

Regards,

Patrick

Dear Patrick,

Thanks for letting me know Akos already sent you a CLA. I was about to send a new one.

I want to contribute my patches in the name of EU Edge so existing CLA is fine.

I made my patch for the bug I discovered. To be honest it’s far from ideal but eventually makes it possible what we need to display flight paths. A complete solution would require a major overhaul of processing and applying KML styles and it takes a lot of effort which is out of scope in this moment.

Also I started working on processing gx:Tour elements. I hope to submit results in the upcoming weeks.

Best regards,

Gábor

Dear Matthew & Andre,

Regarding KML support, I talked to my colleague Gabor the other day, and
I have a couple of questions. I've been on vacation recently, so maybe
I'm not totally up to speed with what's available and what is not already.

For a general overview, looking at your roadmap:
https://github.com/AnalyticalGraphicsInc/cesium/issues/873

do you have a test case to showcase what's working already or not?

can you please confirm that:

  * all items in the roadmap that 'link' to the KML reference are
    'working' in your branch?
  * or is it the items which have a checked tick-mark in front of them
    working?
  * in particular, is the LineString feature working, with all
    attributes (extrude, etc?)
      o after talking to Gabor, I got the impression that the extrude
        feature & styling has some issues.
  * is styling working for LineString, that is:
      o defining a style for it with a LineStyle element will make the
        line of the LineString that style?
      o defining a style for it with a PolyStyle element will make the
        extruded 'wall' of the LineString that style?
  * is your implemenetation working with dynamic geometries, and if so,
    what was the design rationale behind this? as KML files don't change
    after they are loaded, why not use static geometries (which render
    faster in the GL pipeline)?

for a sample that I had working late spring see here:
http://openaviationmap.org/reggeli/

which is a rendering of this KML file:
http://www.diamondaviators.net/reports/files/4427?mode=3d

is there an easy way to see how the same file looks like with your KML
implementation?

best regards,

Akos

Hi Akos, I plan on having a full write-up sometime soon to better explain our plans and roadmap, but you bring up some good questions that I’ll answer below.

Currently the items that are checked in the roadmap are believed to be completed and working in Andre’s branch. Though those items are still pending my review. Andre checks them when he thinks he’s ready to have me review them. Any checked item should be working, but we have separate items for simply processing the geometry versus extruding it. So LinearRing is checked because we read in the coordinates, not because all options for it are supported. Unchecked items are planned, but not at all working yet. The line string is not completely implemented yet, though that is planned. Some of these features depend on improving our Visualizer system, which is a big item on the roadmap I’m actively working on now.

All features should be thoroughly tested by our unit tests in Specs/DynamicScene/KmlDataSourceSpec.js. We will certainly flesh out these tests to cover everything before it comes into master, and any bugs reported will always have a new test added for it so that we prevent future regression.

The styles are not 100% implemented, but will be before this comes into master. I’m not sure exactly which parts are implemented yet or not, but extrusion is not one of them.

DynamicScene is no longer a good name for this layer. We will most likely rename it to DataSource. The general idea is that all data is processed into a common set of objects, which can then all be treated the same by an application, regardless of where the data came from. This also allows the visualization system to automatically determine the best way to render it. If the data is static, then we can use the Geometry & appearances. If it’s dynamic, we use a different rendering path. It will also allow for data to be written out in different formats, as well as be used for additional app features other than visualization. For example, we are working on both a Balloon widget, similar to Google Earth and a data source browser to show the tree structure of the KML document. So whether the data is time-dynamic or not, there are still good reasons for not creating primitives directly and instead go through the DataSource layer.

However, you are mistaken in thinking that KML is not time-dynamic. There are plenty of time-dynamic feature in KML, gx:Track and gx:TimeSpan for example. Here’s an example that uses the time-slider in Google Earth: https://developers.google.com/kml/documentation/us_states.kml Also, the NetworkLink and NetworkLinkUpdate features require the kind of book-keeping that the DynamicScene layer already offers.

I hope that helps explain our thought process. Like I said, I want to post this on the issue or the wiki to give people an idea of where we are heading. It may be too early for you to really make use of what has been done with KML so far, but I expect a lot to change by the end of the month, and if all goes well, we can have initial KML support included in b22 or b23.

Matt

Also, thanks for the KML file, I’ll be sure we include this in our testing so that it renders correctly in Cesium.

Matthew,

Thank for the thorough answer.

The main reason I'm asking all this is that we're also pursuing KML
support, but frankly, we have found some major issues with your
approach, which Gabor has detailed earlier on this list. currently our
conclusion is that our approach is better, with not using dynamic
objects, but creating geometries for the KML constructs.

basically I'd like to dig down in this topic, and find out why the
dynamic object approach would be beneficial, which you have adopted.

Currently the items that are checked in the roadmap are believed to be
completed and working in Andre's branch. Though those items are still
pending my review. Andre checks them when he thinks he's ready to
have me review them. Any checked item should be working, but we have
separate items for simply processing the geometry versus extruding it.
So LinearRing is checked because we read in the coordinates, not
because all options for it are supported.

I see - this clears up a lot

Unchecked items are planned, but not at all working yet. The line
string is not completely implemented yet, though that is planned.
Some of these features depend on improving our Visualizer system,
which is a big item on the roadmap I'm actively working on now.

I see.

from Gabor, and following this thread, I had the impression that styling
/ coloring has considerable issues, in the sense that it's very
difficult to assign different styles to different geometry types, or
different parts of the same geometry (say, the extruded part of a
LineString or the 'line' part of the LineString). is this so? is this
about to improve?

as stated above, we started to work in your codebase, in the hope that
this would have more momentum, but we've found major obstacles that
basically made us stop.

All features should be thoroughly tested by our unit tests in
Specs/DynamicScene/KmlDataSourceSpec.js.

I see

do you have visual samples as well?

We will certainly flesh out these tests to cover everything before it
comes into master, and any bugs reported will always have a new test
added for it so that we prevent future regression.

good to hear

The styles are not 100% implemented, but will be before this comes
into master. I'm not sure exactly which parts are implemented yet or
not, but extrusion is not one of them.

good to know.

as stated earlier, we're really worried about the styling assignment
design (see Gabor's earlier e-mail)

DynamicScene is no longer a good name for this layer. We will most
likely rename it to DataSource. The general idea is that all data is
processed into a common set of objects, which can then all be treated
the same by an application, regardless of where the data came from.

this sounds like a good idea.

OTOH, I think Geometry and Appearance objects also fit the above
description. I wonder what this additional abstraction layer brings to
the table.

This also allows the visualization system to automatically determine
the best way to render it. If the data is static, then we can use the
Geometry & appearances. If it's dynamic, we use a different rendering
path. It will also allow for data to be written out in different
formats, as well as be used for additional app features other than
visualization. For example, we are working on both a Balloon widget,
similar to Google Earth and a data source browser to show the tree
structure of the KML document. So whether the data is time-dynamic or
not, there are still good reasons for not creating primitives directly
and instead go through the DataSource layer.

I see your point, OTOH, I'm a bit skeptical - this sounds too generic,
and thus of limited value. OTOH, I might be wrong :slight_smile:

However, you are mistaken in thinking that KML is not time-dynamic.
There are plenty of time-dynamic feature in KML, gx:Track and
gx:TimeSpan for example. Here's an example that uses the time-slider
in Google
Earth: https://developers.google.com/kml/documentation/us_states.kml
Also, the NetworkLink and NetworkLinkUpdate features require the kind
of book-keeping that the DynamicScene layer already offers.

I see - these are valid points.

I hope that helps explain our thought process. Like I said, I want to
post this on the issue or the wiki to give people an idea of where we
are heading. It may be too early for you to really make use of what
has been done with KML so far, but I expect a lot to change by the end
of the month, and if all goes well, we can have initial KML support
included in b22 or b23.

good to know.

in the meantime, we're working on gx:Tour now, to have it drive camera
movement in Cesium. Will let you know if we have something that actually
works.

our initial goals are to display an extruded LineString, which we've
done already earlier using Geometry objects, and have a gx:Tour object
'lead' the camera through the same path (it's the same aircraft flight
path, visualized with the LineString, and 're-played' using gx:Tour)

Akos