Imagery Layer adjustments/filtering during render

CesiumJS
1

Imagery layer already provides some built in adjustments, like brightness/contrast/hue/saturation, which I believe are performed by shaders in the rendering pipeline. What is the best way to add my own custom adjustments to an imagery layer. For example, I would like to dynamically adjust the brightness of pixels per RGB channel (brighten blue, dim red, etc). Is there any way to add my own shader for a specific imagery layer (and pass parameters to it)? Or perhaps, is there another way to do this without shaders?

Thanks.

2

There is no hook for you to add your own shader per imagery layer. All the shading effects are done when rendering the globe surface, and are performed in the globe’s shaders.

If you would like to modify the Source code to achieve this effect, you would need to create new properties on the ImageryLayer itself, pass these values to the shaders, and modify the shaders.

The values are passed to the globe surface shaders in GlobeSurfaceShaderSet.js, and the shader files themselves are in GlobeVS.js and GlobeFS.js.

I’ve opened an issue to track adding the ability to do this in Cesium without modifying the source code (#6449)We would welcome a PR that is a generic solution, and would be more than happy to provide guidance and review code!

Thanks,

Gabby

3

So the shaders are at a global level, but the parameters are per layer… they must be otherwise I wouldn’t be able to set the brightness/contrast/etc individually per layer. After some more thought I believe that modifying the shaders might not be appropriate for what I need. Ideally I would like to:

  1. Adjust some parameters on an imagery layer

  2. Have the image rendered according to these parameters

  3. Have the imagery data be updated according to these parameters (for example if I create a histogram of the image, I want to see it updated)

Alternatively, I can implement my own imagery provider, wrapping the “real” one, and applying my adjustments to the image data. However I would need a way to tell cesium to re-request the image when adjustments are made, so that I don’t have to wait for a zoom event to refresh the image.

Note, I found a similar post from a few years ago, any update on this: https://groups.google.com/forum/#!topic/cesium-dev/aysBQWn9c88

Thanks a lot for the quick replies!

4

Custom imagery adjustments or processing would definitely be a powerful feature! A few of the Cesium devs have been talking about it in regards to raster analysis. Doing the processing in the shader would most likely be the best option performance-wise. However we’d need to implement a better hook for the custom shader code. This would not modify the underlying imagery texture, and would just change how the imagery is rendered.

For the histogram however, it sounds like what you need to do is grabbing the texture itself like you suggested and then updating the UI.

ImageryProvider.loadImage is where the image itself is returned when requested.

It’s possible trigger the image provider to update. We do that with time for WMTS Imagery. Take a look at the WebMapTileServiceImageryProiver.js source if you’d like to implement your own imagery provider.

5

I’m having some trouble finding the code that triggers cesium to re-fetch imagery… maybe I’m not looking in the right place? I basically need to tell cesium to fetch imagery data programatically, instead of as a result of a zoom level change. If this is done in WebMapTileServiceImageryProvider, could you help me find the lines of code that make the call (I assume periodically)?

Thanks.

6

WebMapTileServiceImageryProvider has an instance of TimeDynamicImagery, which watches for the specified interval and calls that requestImagery function in WebMapTileServiceProvider. Instead of using TimeDynamicImagery to trigger the request, you would trigger it when your UI changes.

Let me know if that doesn’t sufficiently answer your question.

7

From what I can see, the TimeDynamicImagery only calls requestImage inside addToCache(). This is called when the WebMapTileServiceImageryProvider.requestImage tries to access an image that is not in cache, or when preloading the image for an upcoming time interval.
After looking at the stack trace, the actual source of the imageryProvider.requestImage call (for any provider), and the consumer of the returned image, seems to always be the scene render function. Basically it looks like:

Scene.render() -> … -> TileImagery.processStateMachine() -> ImageryLayer._requestImagery() -> anyProvider.requestImage()

The only way we can go from TileImagery.processStateMachine() to ImageryLayer._requestImagery() is if the tile has ImageryState.UNLOADED. So I think my problem is that I need to set the tile states to UNLOADED. I’m not sure how to do that if all I have is a handle to the Imagery Provider and the Imagery Layer. How does that happen for WebMapTileServiceImageryProvider?

For the WMTS example, it seems to happen when the time is being advanced and also when a user manually adjusts the time interval. In the case of a WMS provided imagery layer, it only happens when a new zoom level of a given tile is needed.

Thanks.

8

Maybe create something similar to TimeDynamicImagery which requests a new image periodically, and inside your custom provider requestImage function, set an isDirty flag that will only request after you make a change in your UI.