Custom shader texture with HTML Image Element cannot display

CesiumJS
1

Hello Cesium Support Team,

I’m having trouble initializing a Cesium plane with a custom shader texture, from an html image element. See the code here. I just want to add a video plane in front of my boat, with an image texture. The image texture is from an html image element.. But I got an error from the browser console:
"cesium.js?v=0aece9a6:17174 Uncaught (in promise) TypeError: Failed to construct ‘HTMLImageElement’: Illegal constructor

at clone (cesium.js?v=0aece9a6:17174:18)
at clone (cesium.js?v=0aece9a6:17179:17)
at clone (cesium.js?v=0aece9a6:17179:17)
at clone (cesium.js?v=0aece9a6:17179:17)
at clone (cesium.js?v=0aece9a6:17179:17)
at initializeMaterial (cesium.js?v=0aece9a6:45610:22)
at new Material (cesium.js?v=0aece9a6:45424:3)
at Material.fromType (cesium.js?v=0aece9a6:45440:20)
at cv_video.js:159:37
at initializeVideoShader (cv_video.js:121:7)"

I believe it’s pointing to this line: “material: Cesium.Material.fromType(“CustomImageMaterial”),” I saw some other posts talking about the same issue, and it looks like Cesium is not able to generate an image texture from an html image element?

Alternatively, I’ve also tried these 2 methods:

// let imageTexture = await Cesium.loadImage(defaultImageUrl);

// let imageTexture = await Cesium.Resource.fetchImage({ url: defaultImageUrl, crossOrigin: ‘anonymous’ });

Yet none of them works, still throwing the same error.

Could you tell me how I can solve this issue? Thank you!

2

Hi @Steve_Xie ,

Thanks for your post and welcome to the Cesium community.

Is my understanding correct that you trying to create your own material beyond what the API supports Material - Cesium Documentation ?

From your code it seems like you probably have already seen our resources of fabrics, but in case not here is a thread regarding them that I think is relevant to your work Creating a new material - #2 by Gabby_Getz

I am not sure all the details of generating an image texture from an html image element as you have attempted it. if it is possible for you to provide an example demonstrating your use case in our sandcastle tool https://sandcastle.cesium.com/ that would be helpful in helping debug the implementation or researching how to enhance the api to make your use case possible.

In addition, it might be helpful to provide more context as to why generating the texture from an html image element is useful. And why it is necessary to take this approach as opposed to using an existing texture generation format. This extra context about the use case would be very helpful as well.

Please let us know if you can provide a sandcastle example and hopefully we can provide more clarity soon.

Thanks,
Luke

3

Hi @Luke_McKinstry ,

Thanks for reaching out. My app is listening to a socket server from the backend, to display an html image element. At the same time, I’m trying to read this image as a texture for my custom shader. Here’s a sandcastle I put together. I can’t seem to display the plane (it doesn’t show even if I change it to a regular red material). Could you help me out? Thanks!

4

@Steve_Xie ,

Thanks for sharing more information about your use case and providing a sandcastle example.

I need to further research or run your question by colleagues more familiar with this area of the code. But I did notice this recent PR submission from an open source contributor to expand types of textures allowed for materials and wanted to direct your attention to it. allow further texture uniforms by bkuster · Pull Request #12558 · CesiumGS/cesium · GitHub.

Is this relevant to your use case?

I hope to get back to your question shortly.
Thanks,
Luke

5

Thanks Luke. But this doesn’t seem to help with my question. I’m still having trouble constructing a texture from the HTML Image Element, and it seems the error happens when cloning the Image Element. But somehow it’s not even throwing the error in the sandcastle. Could you provide further instructions? Thanks!

6

There might be some aspects of an XY-Problem here. At least, it’s not entirely clear for me why you are trying to accomplish this with a custom shader. I see that this says “Waiting for video feed”. If the goal is to eventually display some video feed, similar to what is shown in the ‘Video’ Sandcastle (but with some sort of “custom” video source), then this might indeed require further efforts. (And I might not be able to quickly help with further details here…). A high-level approach could then be to look at how that “video” support of that sandcastle was implemented in CesiumJS, and see whether that approach could be generalized or applied to other ~“sources of video data”.

So… maybe it could help to describe the actual goal.

As for the last sandcastle that you posted: I’m not sure what the expectation was behind passing a plane to the PlaneGeometry in

    const planeGeometry = new Cesium.PlaneGeometry({
      // Create a plane with a normal along the positive Z axis.
      plane: new Cesium.Plane(Cesium.Cartesian3.UNIT_Z, 0.0),
...

but that does not work as expected. Again, there is “a lot of stuff” in the sandcastle, and maybe stripping away everything and starting with things that do work (and working towards a goal incrementally) could help to carve out what the issue is.

Here’s something like a “pragmatic fix”, maybe it helps in some way…