Discussion from GitHub: Cesium and three.js

Via issue #6094:

Hi,

I hope it’s fine if I open an issue here while I’m working on a demo with Cesium and three.js combined. ( related to #648 )
You can find a video of a working example here: https://www.youtube.com/watch?v=bnn39nNbDfA
I’ll upload the whole thing once I’ve cleaned some stuff up.

To make this possible, at least for opaque objects, we need to be able to inject three.js render code into the cesium render pipeline. In the example given in the video, I’ve added a callback during the globe pass in Scene.js

// Cesium - Scene.js

us.updatePass(Pass.GLOBE);
var commands = frustumCommands.commands[Pass.GLOBE];
var length = frustumCommands.indices[Pass.GLOBE];
for (j = 0; j < length; ++j) {
    executeCommand(commands[j], scene, context, passState);
}

renderThreeJS({
	numFrustums: numFrustums,
	i: i,
	frustumCommands: frustumCommands,
	scene: scene,
	camera: camera,
	frustum: frustum,
	passState: passState
});

Do you have any suggestion/preference on how you’d allow developers to hook into the rendering pipeline?
Not sure how event handling is done in cesium. I’d usually do something like this:

// first, render globe
us.updatePass(Pass.GLOBE);
var commands = frustumCommands.commands[Pass.GLOBE];
var length = frustumCommands.indices[Pass.GLOBE];
for (j = 0; j < length; ++j) {
    executeCommand(commands[j], scene, context, passState);
}

// then, notify all globe_pass listeners
for(let listener of listeners["globe_pass"]){
	listener(numFrustums, frustum, camera);
}

Threejs developers could then add their render code like this:

cesium.addEventListener("globe_pass", (numFrustums, frustum, camera) => {
	// threejs render code here
	// e.g. copy view and projection matrices, as well as near and far from cesium
	// then call threeRenderer.render(threeScene, threeCamera);
})
  • We’d probably also need to let cesium know of the bounds of the three.js objects.
  • Another callback before the frustum loop is probably also usefull/necessary to give threejs developers the chance to cull and arange the scene for each individual frustum range.

Thanks for redirecting me to the proper channel!

That approach seems reasonable to me. It wouldn’t be hard to add an event between each render stage that sends Cesium’s frameState object, which contains the uniform state (holding the model/view/projection) and near/far ranges.

Those last two points sounds fine as well, as long as it can be done cleanly. The overall goal would be to make it renderer-agnostic.

Do you plan on working this into a PR?

That is the plan, yes. I’m not very familiar with cesium since I don’t use it much myself so I’ll just start to build something that works for me, and hope that the cesium team can help me turn it into something that also works for you as well.