Custom Imagery Provider / Renderer

Summary: I have a custom imagery tiler server-side. This tiler takes large GeoTiff files and tiles them, baking in the raw Float32 value into the RGBA red value. It uses a known min/max to normalize the data and achieve a “raw” imagery tile. Then in Cesium, I have a custom ImageryProvider (CustomTemplateImageryProvider, based on URLTemplateImageryProvider) which passes the image to a “renderer” along with a palette that is confined to the known min/max. The renderer, known as TileRenderer, applies a FS/VS to translate the raw RGBA into a paletted RGBA.

1 (raw) vs. 2 (paletted)

This all works fantastic. It’s something we intend on open sourcing once the custom Cesium modules are more refined to be usable outside of our application.

The problem: Because I’m calling TileRenderer from requestImage inside my ImageProvider, the “paletted” tile is cached by Imagery. I need to move my TileRenderer call toward the end of the stack so that Imagery caches the raw tiles, yet displays the paletted tiles. I’d like to palette the tile immediately before it’s displayed. This will allow me to customize pickFeatures to extract the raw value from the RGBA. In turn, I can allow improved point inspection (not XHR) and also generate “value labels” to overlay onto the imagery. My current method inside TileRenderer involves gl.texImage2D() which requires a canvas/image element, neither of which I appear to be able to access this far into the tile rendering process.

var imageTexture = gl.createTexture();

gl.bindTexture(gl.TEXTURE_2D, imageTexture);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, source);

``

with source being the HTMLImageElement from ImagerProvider.requestImage().

I’ve tried to familiarize myself with the Cesium render process. I see the texture creation in ImageryLayer, along with the reprojection. The Imagery is then cached and recalled from getImageryFromCache. I thought about using finalizeReprojectTexture() to modify the texture, but I still believe this will result in the paletted texture being cached by ImageryLayer.

As I try to work through this, I’m having an issue using textures which are bound to the viewer.canvas context. I’ve created a separate canvas (512x512 / tile size) in TileRenderer for all of the tile palette rendering. For that reason, I can’t bindTexture() the _imageryCache texture unless I use the viewer.canvas. Attaching viewer.canvas to TileRenderer results in odd looking tiles because the 512x512 width doesn’t apply to viewer.canvas. I’m sure I’m missing a canvas where I can perform tile tasks like this without running into context issues, or possibly a queue function where altering the canvas side doesn’t disrupt the map display.

Please note: I’m a WebGL newbie. I’ve used a document.createElement(‘canvas’) inside TileRenderer to render the tiles using the gl.texImage2D() call above. I re-use that canvas for each tile, so I don’t exhaust any limitations. If there’s a more ideal approach, I’m all ears. My FS/VS uses 0->1 normalization (not web merc, so the web merc texture might not be usable here?) though I’d like to use Web Merc coordinates to extract values via pickFeatures.

Thanks in advance for any input you might be able to provide.

Wow, that screen shot looks great! What kind of project are you working in?

I don’t know how to answer your questions, but I’ll ping a few people that might be able to help. Does anyone else have some ideas?

Best,

Hannah

Thank you. The application pictured is a weather data viewer. I’m using Cesium to display NWS/NOAA data transmitted via GRIB2. The old version of my application used Leaflet with a WMS server, but having custom color palettes and non-XHR point inspection are important to me for version 2, which is why I’m using Cesium.

  • Ryan

Hi Ryan,

Cool stuff!

Kevin Ring, who wrote most of the terrain/imagery engine and is often on this forum, is the right person to help you.

For some general info on Cesium’s renderer, see Rendering a Frame, Renderer Architecture, and The Graphics Stack.

Note that you’ll want to replace the direct WebGL calls with calls to Cesium’s renderer API; it is a private API that is subject to change, but that is no problem if your code becomes part of the core engine. This is something we can help you with when you open a pull request. Don’t worry about everything being perfect when you first open it.

Patrick

Thanks for those links. I’ll see if I can work out where to put the code while I wait on Kevin’s reply.

I’ve moved the necessary code toward the end of the render stack, allowing the Imagery cache to only contain raw tile textures.

I’d like to generate Cesium labels using the raw pixel data. I’d also like pickFeatures to retrieve the raw data from the tile instead of making a remote request.

I’m having trouble figuring out which context to run readPixels against.

options.getFeatureInfoFormats = [

{

‘type’: ‘custom’,

‘format’: ‘custom’,

‘callback’: function(x, y, level, longitude, latitude) {

var imagery = activeLayers[0].getImageryFromCache(x, y, level);

viewer.scene.render();

var pixels = imagery.textureWebMercator._context.readPixels({

x : longitude,

y : latitude,

width : 1,

height: 1

}));
console.log(pixels);

return true;

}

}

];

``

However, I keep getting 0, 0, 0, 255 or the post-paletted value instead of an RGBA which only contains a red value (see first post). I also believe the latitude/longitude are wrong and need to be corrected, but haven’t been able to figure that out either.

Any insight is appreciated.

Since I still haven’t heard from Kevin or anyone else, here’s a PR.

https://github.com/AnalyticalGraphicsInc/cesium/pull/4953

Hi Ryan,

is it possible to get a more inside to your custom ImageProvider and TileRenderer ?

Rüdiger

Hi,I saw your wonderful project,and there was a wind layer on the earth,how did you do that?
I wanted to add my own wind data on Cesium,but there was an issue happened.
https://groups.google.com/forum/#!topic/cesium-dev/U3_-gXcu4HA

在 2017年1月4日星期三 UTC+8上午3:23:44,Ryan Hickman写道: