Hi all!
Here is the image of the gun of the ship:
Here is the sandcastle implementation that I’m working with: link
1 Like
@tealminivan
Thank you for sharing such a cool sandcastle demo! Your use-case seems very interesting
I am sharing your question with the rest of the development team at Cesium. I will get back to you with an answer to your questions shorty.
-Sam
@tealminivan
Individual nodes in a glTF model are pickable. However, the glTF model must be created with the primitive API. This sandcastle demo showcases how this can be done.
Note that when you use your mouse to select the propeller, we see the following output in the console.
node: Prop__2_. mesh: Prop
While individual nodes in a glTF model are pickable, you cannot change the color of individual nodes in a glTF model. For more information on changing node colors, check out the following issue.
opened 04:20PM - 08 Jan 15 UTC
closed 09:55PM - 05 Aug 22 UTC
type - enhancement
onramping
priority - high
Is there a way of changing the material linked to a primitive in a ModelMesh cre… ated from a glTF model?
The goal is to highlight the picked node by changing the material's parameters but `ModelMaterial.selValue` is applied over every primitive linked to that material. I think that the best solution is to temporary link the primitive to a special 'highlight' material but it does not have any effect over the rendered model. Maybe there is a better solution to solve the problem so I am open to suggestions!
Thanks!
Your desired functionality is possible with 3D Tiles. Unfortunately, our model tiler does not generate 3D Tilesets that have pickable features. Thus, I can’t really recommend using Cesium ion for this use case.
Sadly there is not a great workaround for you at the moment. I could not think of a quick and easy solution. You may be able to achieve your desired functionality using custom shaders.
CesiumGS:main ← CesiumGS:model-experimental-custom-shaders
opened 03:48PM - 12 Aug 21 UTC
WIP experimental implementation of custom shaders for `ModelExperimental` Openin… g the PR early for feedback and so I can take notes on things that need to be changed.
Description last updated: 2021-08-16: 13:45 EDT
## Overview
This is the core functionality of #9518, though there will be additional PRs in the future to add the metadata capabilities.
Custom shaders allows creating custom vertex and fragment shader code that can be applied anywhere a Model (for now, `ModelExperimental`) is used.
Note that this PR only implements the basic implementation. A future PR will add metadata to custom shaders, though this requires more architecture elsewhere in `ModelExperimental`-related classes.
There are two ways to try this feature right now:
* Load a tileset that uses .glb tiles via `3DTILES_content_gltf`. This requires `Cesium.ExperimentalFeatures.enableModelExperimental `. B3DM and other tile types are not supported yet, we've only wired up `ModelExperimental` to `Gltf3DTileContent` so far.
* Load a GLB or GLTF file via `ModelExperimental.fromGltf()`
## Sandcastle Examples
EDIT 2021-08-25: Due to recent changes the following examples won't work without a small tweak to the vertex shader, the new signature should be:
```glsl
void vertexMain(VertexInput vsInput, inout vec3 position) {
}
```
* [Simple example of tinting the baseColor texture](http://cesium-dev.s3-website-us-east-1.amazonaws.com/cesium/model-experimental-custom-shaders/Apps/Sandcastle/index.html#c=fVZrT+M4FP0rFp9SqaQtA+wOLy1TKFPUUqYEGEaVGDdxG28dOxM7lLKa/77HzqPloZFQiq/v69x7fO0u0zxP/L6SfsRmNBfmNAyZ1oFaMEmOyWSLrS7j6UXIR/yyf/vS71zxvu7L8V7Y7e/3F+n3u+7lZx9Kv6KLhVVaPCR38ejsjg9X7dXDzrflIBgnVy+nZnQ/3LvikN1fxsOz087wvpf8uOjzQfcy/QFnw2C+NwwWup+IOHLrh9XoLNyFbDX63vZ/nF1/jdJkl/Ue8ufwZi53754H+zvXsWJ3s71g/uvsurezuv3M9r9Ntg4nciK7Bbjz55RlPGHSUNFj1OQZ0z6TdCrYUEVMbO4Dssly5syfaEaeOFuyDFLJlqT0d+dk3mQrdOuugiGXLJtsNWo7bWhmAjiFKbzPVJZQGTJfqqW31gpzbVRyE9PobYzuxo7330QSkktuvegD4paEtFrk/JkmqWBEzYhmxnA5J7R0WujkjwY51CaEPFGRY12n16w24C0YnY0OCI0iovM0VZkhCEgMe7YVq+NXBmaVwlGZ7m2xGUDm9waj06D0+9v9Fl9E6OZZhjITlRqupHb+dYE+QSeIJ7g2iD79l4WGGEXCmMo5IxFLmYwsPCVJrtksFxIkbZDS73B01u89PA5Pg/Nx/3RAvJLLDbJNvIQatBet3T4pa1PFhEDweWzr1iAmZoTLNDekNlhyIciUkTRj26lKc4GNCFITV1UBhAz+rO1cBD3iTalmXSVU1iTS9lwgySDmmuCvSLwoKpeuW7UbyGiaipUVpkqb7TRT9hzaNZvNUA5dYh2fXw9Ou+drsID4B1gXDMxE3iBGjcvlrEOIw9hHeqzMda2xEd3LGI02c21YLHrB05RFfg1urlTkoDnTKM/gpbLwbeq2wQcf0dueQf9NByvC2NzMUtV4nBdNKPh4/WVMqIzI7dWgH5TqVlaeNr22CakIbe8s5UpFZ1RwDgcH2YvHXkbnrnPE9jCyTKvK4Ud8NkP3XLxaSEUa0yZhTyxbmdg1SkDHlmOOeqI4NliVhZs0Nf7BptR32VSQx3YLDVlaHnCbtHMpXbaC6JRijNRlf39WPDuyCjWSOF8gobI0jplILQHhSytMphA4dcOGBQT0qmhHgKYdkJ+FOPxU7g0x4bw792/fHZIn7X6bhVKVaqOaNBlD52UtP3RjwH5+VjhrrrmRZNu1wEEvCt3SKbMty1ynqdZ5go4Yh0Jg1NLM1V2SZYzPK3LYotSUmKGndq6/R6Z4VO86bL1yUaCbVei4VFiGL8mjK/+wSrrKvrExjRO6YDrmM0OWdjeherHm0tebL0CHuYtboDiAU8AubF0JYz3FFWAjjS++BAr63lv+NQ4L/ZlQ1DhvPTj6Aj+wpFPtwYf/jIHQ9vf39yttZCZA3KFN55h0/DYUXIJW8srjhpo2LPXafqf5Kkxxc5VO7Qxzk8+aYbysmBBquT3PGJMbuFTG51xSUZyt43eH6nBD17pEuaDEn71XhgXPkNFeE+j+tp9Ow373KpjvDuuHXooIzRpqY4OZE/m7AIjL2V3xvg4xPX07j2/TCP59XI7n4J0ZoCh2sHqzXIaW4F5JhM0b3cdoKS9GPBaKq3iy1STeuwcBOlLfyA3SIp12u21T+b1+KhguGPwB1evc8HDBCXvCkEVunk1h8xXhfj6dBYWxt+bqNaLZZ4Pt30gycq8yEZGAZRkOA7lHK0kPKaIZ9gfc6Pg7jswYlWpa8jbP1hMNL8gx0yrPgMnS+1QjYD/y9nZ3d/7abTTryCmOsx1BNvLre6sSYkzjDRYalfnk3E5XB764jdHYNQHtDNQUk6xwUGxsduDg1cp1GvOuLOofylgW2yqWai9KJYFab2w1t460WQl2Ur2H/uGJezOhKJ7vtwzDywyU0a1pHi6Y8UO8V0qqHrU2TY8i/kR4dPzBe5KEAtXCDsa2uOEvoM/JUQv670xxfu0lPUK5BF1ZtbhzMiiEvu8ftbD82NIoJaY0e+P5fw) (Updated 2021-08-16 13:40) -- this shows how the `MODIFY_MATERIAL` mode can be used to update the textures
* [PBR Shading the Ocean](http://cesium-dev.s3-website-us-east-1.amazonaws.com/cesium/model-experimental-custom-shaders/Apps/Sandcastle/index.html#c=fVdhc+I4Ev0rqnw5c8sYwpBcbSYzdRkIWVIQsoyTTKaoygpbYC2y5bVkCLma/35Pkm1ISE1VCpDU3eru1/3U6THFi8QfytSP2IIWQl+EIVMqkCuWks9kdsS21/H8KuQTfj28exke3/ChGqbTk7A3PB2usu/3vevffQj9E12tjNDqMbmPJ/17Pt62t4+dPzejYJrcvFzoycP45IZj7+E6HvcvjscPg+TH1ZCPetfZDxgbB8uTcbBSw0TEkV0/bif9sIu97eR72//Rv/0jypIuGzwWz+G3Zdq9fx6ddm5jye4XJ8Hyn/7toLO9+52d/jk7+jRLZ2nPBXf5nLGcJyzVVAwY1UXOlM9SOhdsLCMm9s8Rss4LZtXXNCdrzjYsx27KNqS0d2/3vNlRaNc9CUWesnx21Kj1lKa5DmAUqrC+kHlC05D5qdx4O6mwUFom32Iavb2jt3fi/W+WElKk3FhRZ8QuCWm1yOUzTTLBiFwQxbTm6ZLQ0qiTKZ40fKhVCFlTUWBdu9esDmAtmPQnZ4RGEVFFlslcE1xINHs2GavvrxT0NoOh0t07dxhgzx+MJhdBafen/XafuKFX5DnSTGSmuUyVta9c9AmQIJ7gSuP2+d8s1ERLEsY0XTISsYylkQlPpqRQbFGIFEXaKM2OJ/3h4PFpfBFcTocXI/KBeAnVwBR4fvhSJqS6CBuCL2OTrAbRMSM8zQpNaoUNF4LMGcly9iGTWSFwEGFXx1Uq4HcOe0Z3KYIB8eZUsZ4UMm+S1AAtVIMEMVcEf85bl0meWohqM9ijWSa2ZjOTSn/Icmmaz6zZYoEcqDLC6eXt6KJ3+SrEX4R1xVCO8BvVUMdlfVYhtsPYh3us9HUnsXe7lzMa7fvaMLGoFc8yFvl1cEspIxuaVY2KHFYqDd+4blA9e6+mTeP5b3CrqsT4pjeyjsdaUYSiCG+/TglNI3J3MxoGpbjZK1tM7XRCKkKDnamzUtAquUJDt8B78TTI6dIiRwyGkSmvKh1+xBcLoGfvqzepyGLaJGzN8q2OLVACMiYdS+QTyTGXVV5YeqnjH+3v+nC7CnhqDgDHxlQBNy5bg6n1VRCVUTBHnfTD9vAMSzkxklhbKEFpijhmIjPlB1tKgoxCROnaBgEAKQdGAMjOyF9uO/xYno1Bat69/Tm0LbJW9rvphCpXGxW55Ay4p/X+J9v55uOvKs660iwLGbBW6G2X5pbKmAEstzhTpYoEeGgbhQC70txmPSWbGB+vSsMkpS6IBRA1VH4YmeRRfWpjG5QLF92iio6nEsvwJXmy6R9XTlfeN/YIOKErpmK+0GRjThOqVrtK+uPbV0QHqgXxu/abI2yna1MYqzlY39w0vfoaSMh7b6uv8cnJL4Sk2lobwNBX2IEmnSsPNvxn0EHbPz09fS1tXRoblz6TY78NIaVZ5rX94+YrS5VWHdXIlDySCq5GDhEMRfeRXBbLuIKrzkat9GA2DrUimq9s3CRheF8FD3dl4P0NRigfART/Jqb6X8oUkOJ4mxu16RsJKtM4JTyhqHeX4boGcGcqNdkyc47n0GCKc3TAwZWqtmlpXkgJ5/ByGs9Ty+iLwrDXHkgCyXAU8fmAGz7tydlsV4JmB4lunzRJ9dmt0lwbsQk175ixzJ89YGQEPzZ3yB3o1F3iVKp7uo3mngO/MlDh5/Tr4Jq1y4032nXGqNY5nxe6atzg8ntvMpn2n9quZQuQO/gWxCu2Zopw7Wtz6lrL37OgMGWFuk5fhxRruHQo6IN9elLm0VP7bZWO0XuO1TBKcepo04FYvsJNAhvoRPhTHtjIXvfImh3jZrx6Xgc98m/bjhnHj65dFmvbXiduYcept122Zp33LURSG4Q6DtmPNrfFugFz3XfM1RA573Htbw6kttF2jv6G+jiBpr30AGbHtrP0p8MNM6adVH0VYh7wzYRxl0VQ8YHOJbhUj8ACZlTwFkUamgR6JbntD6Y+HstyvsPM63yeHTWJdzDXWoYpB8sGaZHjdrttXPm5m3g1Fwz2TIu88g3zN16NNQCHb55xYX8Ytl8f+4FT9nb8e4vbzPRrsJ2kjDzIXEQkYHkOgicPDD0+gIsoRfMFtI79jqUPPP5yXrZ5ke/eaPwjNGVKFjliMpR9oXDhMPJOut3Of9Bl9c0Znqiqql5PYtUmBg9b5DL3yaWZF2zwjnjQgbUlbd51RVHHzoA72Efg7NXKIg1yLJP6izSWyTaCpdiLlEkgdwdHzaNzpbeCfanG+v+CQs3oj6R4vt/SDIyKklGteRGumPZDjN1lwZ639lXPI74mPPr8zr9FJBTIFk4wiohv/AXl8+W8BfkDVXSUGTsnSJegWyMWH38ZuU3f989bWL6vqaUUc5q/sfx/) (Updated 2021-08-16 13:40) -- this demonstrates how you can set the PBR properties properties like `material.specular` or `material.roughness`.
* [Custom glTF attributes example](http://cesium-dev.s3-website-us-east-1.amazonaws.com/cesium/model-experimental-custom-shaders/Apps/Sandcastle/index.html#c=fVZtTxs5EP4rFp82d7AJFHoqB+hyCeGCEkLpAqWKRJ1dJ+vDa29tb0I48d/vsfeFAFUltMHjeZ9nZtxjhhdZOFQyTNicFsJ245gZE6kHJskxmW6x9Xk6O4v5hJ8Pr5+Guxd8aIby6iDuDT8OH/KvN73zTyGYfiRnD47p4S67SSf9Gz5ed9Z3e59Xo+gqu3jq2snt+OCCg3Z7no773d3x7SD7djbko955/g3KxtHiYBw9mGEm0sSf79aTfrwP2nrytRN+61/+k+TZPhvcFY/xl4Xcv3kcfdy7TBW7mR9Eix/9y8He+voT+/h5uvXnVE5lrwzu9DFnmmdMWioGjNpCMxMySWeCjVXCxOY9Qra6YF58STVZcrZiGlTJVqTSd+NpwXQr9ueegiCXTE+3Wo2csVTbCEohCu1zpTMqYxZKtQpeuOLCWJV9SWny1kZv4yb4byoJKSR3Wswh8UdC2m1y+kizXDCi5sQwa7lcEFopLXmKewsfGhFCllQUODfubdcX0BZN+pNDQpOEmCLPlbYEBolljy5jjf1awK5zKKrcvS4vI9DCwWjSjSq9z/63/MJCr9AaaSYqt1xJ4/WbMvoMlSCB4MbC+uxfFltiFYlTKheMJCxnMnHhKUkKw+aFkABpq1I7nvSHg7v7cTc6vRp2R2SHBBm1qCnquXNSJaQ2BILgi9Qlq0VsygiXeWFJI7DiQpAZI7lmO7nKC4GLBFSb1qmA3xr6nOxCRAMSzKhhPSWU3ibSFVqYFolSbgj+Sm/LTHLpS9SoAY3muVg7Yq6M3cm1cs3nzmw+Rw5MFeHV6eWo2zt9FeIvwjpjgCP8BhqauLzPJgY5TkO4xypfXzg2rAea0WTT15aLxTzwPGdJ2AS3UCrxoXnRpNDQUkuEznVX1cOfYdo1XvimbjVKnG92pZp4vBZDKEB4+fcVoTIh1xejYVSxO1rVYuZFJqYidrVzOKsYvVAJNHQLvBf3A00XvnLE1TBx8KrTESZ8Pkf1vL2GSEWe0m3ClkyvbeoLJcDj0rFAPpEcZ6z2wo+XJv7RJjWE23XAV+4C5Vg5FHDnslcova+CmJxicjRJf98egZtSJRvJvC5AUDkQp0zkDn7QZRSGUYwoy7ZBAKhUWYwIJTsk30ty/KG6G2OoBTf+36FvkaXxv9slU+1qa2MejekD831ROl7IxLePM++ovz+ShGv47iN8FV6pYi4UtWRFl25sdsJd8hsBHIMPYQf/1QbDR6C/HGxukjo5zQA5+ZK9372LQSfsbBP/cSpL5mf3+V5nvsG+n4sOPgigKnzb5MxBSHvkUWOKDAixPq8C855qjwNJVik+r8DqytRAdA6MueXyPteKJ82tz/agOpT5ntf55lLhGD9l9z5j49rp2vvNEgwJjWOe+GUm1mBxaEfpS7DaFPnVLOMyMSTzi2OuuCjjVrLRUshYZRkyaXU5B2Kqk3JgFagk5ljM7RprhMcPTBvXNx5fmANJUY2tCkwzqteoZhVMSK3VfFZYDAh3E0OX5vF9rBQMVOVsuq2pwLFX8/Zaq2KRukXg0bL39rpu4OMaDfsODeVnEwxT+VyuZCxlv9pDA7dY6EbydZ5AXYileIpS2xH2k5utwRwZclALqtxvbvIQ06VaiHgklEidbm2T4N1DAEBuNnGLtMlup9Nxrjy/PBEsFwz6XBCvfMODBVhfIo3wLXAubL4e/M+HflQKBy/wuIQ1V3XXjhPJyK3SIiER0xr4I7cMu28AFwF09wOw7IZ7fjZiWqpZ1aaFfhlqeDleMaMKjZjciukaGBwmwcH+/oc/Oq3txnKODqoHwevVVRMxqfH2iq3SITl1A9YHXy5k1LHRZN0gNBRwKxWUF5sVOHx18pXG0KuS+os0Vsl2jBXbk1JZpF4utra3joxdC3ZSv4P+4pl/KyEpQRi2LcOLDJAx7VmB3rBhjHdKhcyj9qboUcKXhCfHP3lHklggW7jB7BZf+BPgc3LUBv87UQxM158TpEvQtWNLd09GJTEMw6M2jj+XtEoJNNUbzf8D) (Updated 2021-08-16 13:40) -- in this case I made a model with a custom attribute `_BARYCENTRIC_COORDS`. This is accessible via `fsInput.attributes.barycentric_coords`.
* [Custom varyings](http://cesium-dev.s3-website-us-east-1.amazonaws.com/cesium/model-experimental-custom-shaders/Apps/Sandcastle/index.html#c=fVf7c9o4EP5XNPnJ9Ighr941TTuXI5CSgZKmTtJ0mEmFLbAP23IlmUdu+r/ft/IDSNrOJAav9r3frpaO0FGeuH2ZuoGY8jw2574vtPbkXKTsHRvvifVVOLn0o1F01b996h98jPq6n96c+J3+6/48+3LXuXrjgul7cDknpvlDcheOLu6i4bq9fjj8tBx4N8nHp3Mzuh+efIxAu78KhxfnB8P7XvL1sh8NOlfZVygberOToTfX/SQOA/v+sB5d+MegrUdf2u7Xi+sPQZYci95DvvI/z9Lju9Xg9eF1KMXd9MSbfb+47h2ub9+I15/Ge2/H6TjtFMF1V5lQUSJSw+Oe4CZXQrsi5ZNYDGUg4u1zhGxULqz4giu2iMRSKFBTsWSlvjtLc8Z7vn3vSAhGqVDjvUYtpw1XxoNSiEL7VKqEp75wU7l0Nlx+ro1MPoc8eG6js3Xi/DdOGcvTiLToU2ZfGWu1WHfFkywWTE6ZFsZE6YzxUmnBkz8a+FCLMLbgcY732r1mdQBt3uhidMp4EDCdZ5lUhsEgM2JFGavtVwJmnUFR6e5tceiB5vYGo3Ov4Pph1RdPxLuGgxv/F49BBD+QlZ6SiSezWttdwbqrrdACNzu5UqgVk5mJZKqtk7pIYYJyMieGWoQw+Vf4hhnJ/JCnM8ECkYk0oBzJlOVaTPM4BdIbpdrh6KLfe3gcnnvdm/75gO0zJ+EGwAAo9t+XWa0MgRBHs5Ay3mAmFCxKs9ywWmAZxTGbCJYpsZ/JLI9xEIBqwiqf8BtRW9lZ7PWYM+FadGQsVZOlhJZYN5gXRprhr/C2KEeU2jrXakDjWRZTwlgmtdnPlKQOpncxnSIHuozwpns9OO90d0L8TViXApiG34BUHZf1Wfsg+6EL90Tp64Zjy7qjBA+2fW1QLHoeZZkI3Dq4mZSBDc2KBrmClkrCJdepqqc/awzqXvdZ3SqUkG9mKet4rBbNOJB8/c8N42nAbj8O+l7JTrSyT/VGxuexT7UjnJWMVqgAGloO3sePPcVntnKMahgQvKp0uEE0naJ61l5N5HEW8iYTC6HWJrSFisFD6Zghn0gOGau8sDOqjn+wTXXhdhXwDR2gHEtCQUQuW4Wp9TVmOuMYP3XSX7aHQ6OuYGOJ1QUISgJxKOKM4AddWmKi+YiyaBsEgEoVxfBQslP2rSD7R+XZEJPRubNf+7ZFFtp+NgumytXG1lCjyiFlmI4YrErGmoUIqgRqNUVsIVGAQgqcaPlpLLlhnz90B4PHD93+5QcPM/XQbb/9xbzBaUVxyBun7babzD62lTSaGzdLVbWvCZ8L28MaSYoxIwPb6pQqov6xggWFPNtq7JSiUFG4vOQLuifa7gF7xdA6zpHbxrfKqrtCpxaTvHJACbRHuqn0H+xZAKSyYP5Bj28VSuo+tRcBQR0BlCBt6UwQ3JVNLtc6T4BmYzEQ44LjymI2ZcsQj53GIkjV7TRFP9Bt+hIXMgrqU4uMXvlSYGNaYSNKJV79p+TRZmxYOV15X8PlWf5OkIi2e3hSpvHQpnE3cxZ2E6AIAqU9lxujokluMG/oxIdDKvIffSlVoN9uW0qiFOQkkyndQO/o3aF/EnNXTavYXTfKL0+NHWGfIPKOEuAbZ1fTK8LpLjeN2kdyxa4GfKIdim/fatnlTLieg0UnUppQG5GB87DEwZaWXSFC/nMh8B/R40WnVKL1BKuRQilY1d3TaFY4PCD7f9HjCETy8IUOJfNZSDdwqaTkP/gFezVJdywelWEi4I3tdqmmsE2hbHfCOP1RLGBYwewi52rkR7h0d95mAcy5WIG6wLkZQJQuQWeap7aHnRJ423ubiylUrj9YCQuwjfeazHmx9qF49d7VYC120G7bmv/YLIQmigX0Ichd37CeotEXACh8c8iF7V3RfhxdeIWwsxml17BGyyHNolEq2L1UccA8oRSaj90THntwEV1OH0DFgXtoLzFca3JSzqhcbW4f/E64EVrmCjERtM41DPYD5+T4+OhPlL+2nGF8VFNwd8eoiDSyVe4bqVzWpZvQBl9sTqhzrcnQjaV5IkoF5cTfqsDpzputNG6nMqm/SWOZbGIs2Z4kwX1zsNfcO9NmHYv31db7d5TYzRhJcVy3ZQT2b0BGtya5PxfG9bFQlsg9a22LngXRgkXBu5/8amB+jGzhBJds/Dl6Anzen7XA/0IUnUsL1Qjpivma2MKD94OC6LruWQuvP5c0UsaYSc80/w8) (Updated 2021-08-16 13:45) -- this example shows how to define additional varyings, set them in the custom vertex shader and use them in the custom fragment shader.
## `CustomShader` Documentation
>Update 2021-08-23: The latest documentation for custom shaders is now in a [README file] on this branch. It is temporarily in `Source/Scene/ModelExperimental/CustomShaderGuide/` for now while this feature is experimental.
CustomShader is a bit involved, so here's how it gets used. This may become the start of a user guide for custom shader, so I'll keep updating this as I update the PR.
Constructor:
```js
var customShader = new Cesium.CustomShader({
// Any custom uniforms the user wants to add to the shader.
// these can be changed at runtime via customShader.setUniform()
uniforms: {
u_time: {
value: 0, // initial value
type: UniformType.FLOAT
},
}
// custom varyings that will appear in the custom vertex and fragment shader text.
// the user is responsible for setting these in the vertex shader and using them
// in the fragment shader.
varyings: {
v_customTexCoords: VaryingType.VEC2
}, // optional
// configure where in the fragment shader's materials/lighting pipeline the
// custom shader goes. More on this below.
mode: Cesium.CustomShaderMode.MODIFY_MATERIAL,
// either PBR (physically-based rendering) or UNLIT depending on the desired
// results.
lightingModel: Cesium.LightingModel.PBR,
// Custom vertex shader. Right now this is a function from model space -> model space.
// VertexInput will be documented below
// NOTE: which coordinate system is used here may change (e.g. could become model -> view space
vertexShaderText: `
vec3 vertexMain(VertexInput vsInput, vec3 position) {
// code goes here. e.g. for a no-op:
return position;
}
`,
// Custom fragment shader.
// FragmentInput will be documented below
// Regardless of the mode, this always takes in a material and modifies it in place.
fragmentShaderText: `
void fragmentMain(FragmentInput fsInput, inout czm_modelMaterial material) {
// code goes here. e.g. to set the diffuse color to solid red:
material.diffuse = vec3(1.0, 0.0, 0.0);
}
`,
});
```
### Custom Shader Modes
EDIT 2021-08-16: I removed `BEFORE_MATERIAL` and `AFTER_LIGHTING` to simplify the interface.
NOTE: the list of modes is subject to change. We might decide to make further simplifications.
The custom fragment shader is configurable so it can go before/after materials or lighting. here's a summary of what
modes are available.
| Mode | Fragment shader pipeline | Description |
|---------| -----------------------------------|----------------|
| `MODIFY_MATERIAL` (default) | material -> custom shader -> lighting | The custom shader modifies the results of the material stage |
| `REPLACE_MATERIAL` | custom shader -> lighting | Don't run the material stage at all, but procedurally generate it in the custom shader |
In the above, "material" does preprocessing of glTF textures, resulting in a `czm_modelMaterial`. This is mostly relevant for PBR, but even for UNLIT, the base color texture is handled.
### `VertexInput` struct
An automatically-generated GLSL struct that contains glTF attributes. In the future we may add automatic fields. in a future PR, metadata will be added.
Right now an example struct looks like this:
```glsl
// this struct represents the raw attribute values.
struct Attributes {
// required semantics
vec3 position; // model space position. always present as glTF POSITION is required
// optional semantics (added depending on what's in the glTF)
vec3 normal; // glTF NORMAL
vec3 tangent;
vec3 texCoord_0;
// etc.
// custom attribues
vec3 custom_attribute; // corresponds to glTF attribute _CUSTOM_ATTRIBUTE
};
struct VertexInput {
// raw attribute values
Attributes attributes;
// in the future we may add another struct for derived attributes (e.g. positionEC)
};
```
### `FragmentInput` struct
This struct is similar to `VertexInput`, but there are a few more variables for positions in various coordinate spaces.
```glsl
// this struct represents the raw glTF attributes. The varyings required to make this work are handled
// automatically.
struct Attributes {
// required semantics
vec3 position; // Raw model space position. always present as glTF POSITION is required
// optional semantics (added depending on what's in the glTF)
vec3 normal; // glTF NORMAL
vec3 tangent;
vec3 texCoord_0;
// etc.
// custom attribues
vec3 custom_attribute; // corresponds to glTF attribute _CUSTOM_ATTRIBUTE
};
struct FragmentInput {
// raw glTF attribute values interpolated (but not normalized) from varyings.
Attributes attributes;
vec3 positionMC; // model space
vec3 positionWC; // World coords (ECEF). Low precision.
vec3 positionEC; // Eye coordinates
};
```
### `czm_modelMaterial` struct
This one is a built-in, see the [documentation comment](https://github.com/CesiumGS/cesium/blob/model-experimental-custom-shaders/Source/Shaders/Builtin/Structs/modelMaterial.glsl). This is similar to `czm_material` from the old Fabric system, but slightly different fields as this one supports PBR lighting.
This struct serves as the basic input/output of the fragment shader pipeline stages. For example:
* the material stage produces a material
* the lighting stage takes in a material, computes lighting, and stores the result into `material.diffuse`
* Custom shaders (regardless of where in the pipeline it is) takes in a material (even if it's a material with default values) and modifies this.
## To Do:
- [x] Document usage more thoroughly in this PR description
- [x] Add development sandcastle
- [x] Where should documentation go?
- [x] Add unit tests - the basic regiment of tests are implemented, I'm trying to keep them updated in each commit.
- [x] uniform and varying types should be enums, not raw strings
- [x] `uniform sampler2D`s should be added. This requires some resource management, as the custom shader will need to fetch images and create `Texture` objects
- [x] Discussion: if we allow texture samplers should we also allow cubemap samplers? though this should go beyond the scope of this PR. (see https://github.com/CesiumGS/cesium/issues/9741)
- [x] Consider adding a built-in uniform for implicit tile coordinates (when `3DTILES_implicit_tiling` is used) EDIT: We're holding off on this, I opened #9736 to track this feature.
- [x] Discuss whether the custom vertex shader should be in model space, view space, or some other space. Right now it's a callback that goes from model -> model space.
- [x] Implement various reference frames (model, cartesian, cartographic, and view space) EDIT: cartographic will have to wait. Opened https://github.com/CesiumGS/cesium/issues/9735 for this.
- [x] Discuss the shader modes more with @lilleyse and @sanjeetsuhag about the shader modes. We're considering changing the list of them. For example, instead of `AFTER_LIGHTING`, we could just have an optional `afterLighting` GLSL callback. This way you could both modify the material _and_ post-process the lighting EDIT: We decided to skip this for now
- [x] handle heterogenous data: what if a tileset has multiple types of models, some with normals, some without? This should output a warning and a sensible default should be used.
- [x] Update unit tests. Recent changes added new classes and significantly changed the behavior of `CustomShaderStage`
However, we are still developing this technology and I am not 100% confident that it can be used for this purpose. @sean_lilley posted some thoughts and suggestions on the following issue.
opened 08:03PM - 13 Aug 21 UTC
category - model
category - custom shaders
Custom Shaders (issue #9518 and initial implementation #9725) allows applying us… er-defined vertex and fragment shaders to glTF models as well as 3D Tiles. This is new behavior, as the old materials system was not compatible with 3D Tiles.
While the implementation is still a work in progress, we still want to gather ideas of what exciting things will be possible with this new feature.
Here are a few ideas we've seen already, though more are welcome!
* glTF supports storing [application-specific semantics](https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/README.md#meshes) (custom vertex attributes). For example, a mesh could have `_TEMPERATURE`. Custom shaders provides a method to access these values and use them in the shader
* (requires glTF `EXT_feature_metadata`) Make a tileset that includes simple polygons with metadata labeling them as "grass" or "building" etc. Then use custom shaders to procedurally apply textures
* A user could apply a mask/tint to areas of imagery data in more nuanced ways than with styling. This will be even more useful once `EXT_feature_metadata` is available and the shader can use metadata to mask where the shading should be applied.
* If we include cartographic WGS84 coordinates (caveat: 32-bit precision limitations on the GPU) you could easily overlay a lat/lon grid over 3D tiles.
* @srothst1 had a couple interesting ideas for visualization:
* Weather visualizations on a tileset with terrain/imagery. For example, maybe visualize wildfires or other extreme weather events in the news. This would probably be a good use case for `EXT_feature_metadata` feature textures or feature ID textures.
* We talked about how you might represent [The Valley Curtain](https://christojeanneclaude.net/artworks/valley-curtain/) in CesiumJS. While certainly you could do something simple with a glTF and animations alone, with a custom vertex shader you could simulate the curtain flowing in the breeze (perhaps with procedural noise or something)
Thanks to @mramato and @srothst1 and others for some of the above ideas.
@mramato mentioned a couple more ideas in slack, could you expand on them in more detail here?
I recommend checking this out. Let me know if you have any other questions or comments!
-Sam
1 Like
