Community support for WebVMT geotagged web video integration with CesiumJS?

[rjksmith]

Would integration with WebVMT for interoperability with geotagged video users on the web be valuable to the Cesium community?

What is WebVMT?

Web Video Map Tracks (WebVMT) is an open format for synchronising location with video in a web browser, which was published jointly by W3C and OGC in September 2023.

OGC Testbeds have played an important role in guiding WebVMT development in recent years including accessibility, tracking moving objects, camera orientation, sensor synchronisation and data aggregation. Details and demos can be found at webvmt.org.

Comparison with CZML

WebVMT and CZML have different aims:

• CZML is designed specifically to support Cesium.

• WebVMT is designed to support timed video metadata, including geospatial data, in a web browser.

WebVMT is a metadata-focussed variant of WebVTT (Web Video Text Tracks) which is already widely supported by web browsers to facilitate adoption by W3C and browser implementers.

Many of Cesium’s features are supported natively by WebVMT - such as paths and polygons. WebVMT also supports extensions, so CZML JSON data can be embedded within WebVMT to synchronise unsupported Cesium features with the web video timeline. For example:

NOTE Attach CZML data to 5 second mark in video timeline

00:00:05.000 -->
{"sync": {"type": "com.cesium.czml",
          "data": <CZML_JSON_DATA>}}

Who can use it?

Users of drones, dash cams, body-worn cameras and smartphones can share geotagged footage online with their map of choice. The Get Started guide shows content creators how to export data to WebVMT using free community tools.

Search video by location

HTML integration ensures that geospatial data are accessible to online search engines regardless of video encoding, so web users can quickly find video content by location.

Wider web map API support?

The WebVMT Javascript engine already supports several open source web map APIs including:

• LeafletJS
• OpenLayers
• MapLibre

Each API offers unique features which allow users to choose the best solution for their use case, and the inclusion of CesiumJS would widen this range further. Both Cesium and WebVMT ecosystems could benefit from interoperability, if there is sufficient community interest.

Community feedback

Please show your support for this initiative by registering your interest and use cases in the comments below. Your feedback is critical to the success of this effort.

[rjksmith]

Powder tracks

Greg Beatty’s blog post from May 2013 outlines a skiing use case for geotagged video with Cesium.

Using a GoPro camera and GPS tracking Android app, this demo showcases how Cesium can provide geospatial context for video footage. HTML5 video frames are copied into a WebGL texture to synchronise Cesium with the video timeline.

WebVMT now offers an alternative synchronisation mechanism that leverages the native HTML5 video player and eliminates the graphical duplication step. This lightweight approach can reduce energy overheads and extend the battery life for mobile devices.

[rjksmith]

Paragliding training

Sarah Chow’s blog post from June 2018 describes a paraglider video use case with a telemetry overlay that includes Cesium maps.

Location and sensor telemetry from GoPro footage are used to display a video overlay providing operational context for critical analysis to train pilots. Telemetry are exported with gpmf-parser and displayed using WebVfx to synchronise with the video timeline.

Telemetry can now be exported to WebVMT using the Get Started guide and synchronised with the native HTML5 video player. Location and sensor telemetry can be linked to the video with a metadata TextTrack and delivered using either VTTCue or the forthcoming DataCue via event handlers.

    <video src="paragliding.mp4" controls>
      <track kind="metadata" src="telemetry.vmt" />
      <!-- Show message if video tag not supported -->
      Your browser doesn't support HTML video.
    </video>
    <div>
      <!-- Telemetry display elements with event listeners -->
    </div>

[rjksmith]

W3C Breakouts Day 2025

Join us online at the W3C Breakouts Day on 26 March 2025 to discuss ‘Synchronising Sensors With Geotagged Video.’

This event is free and open to anyone who wants to raise an issue about synchronising sensors and geospatial data with video in a web browser or find out more about this topic.

Full details at W3C Breakouts Day 2025

[rjksmith]

Many thanks to all who participated.

A summary has now been posted for those who missed the session.

[Gabby_Getz]

Thanks @rjksmith! Does this related at all to Full motion video support (FMV) · Issue #11287 · CesiumGS/cesium · GitHub?

[rjksmith]

@Gabby_Getz The short answer is: yes.

The longer answer is: I believe that issue #11287 refers to a commercial solution (ArcGIS Pro) for synchronising full-motion video (FMV) with location data. By contrast, WebVMT is an open format for synchronising FMV with geospatial and sensor data in a web browser.

I demonstrated WebVMT’s heading feature using that same video data in the session I chaired at the W3C Breakouts Day last week. A description of the demo and a couple of screenshots are included in the session summary.

Please let me know if you need further details.

[Gabby_Getz]

Awesome, thanks for the explanation @rjksmith!