Just a quick heads up that we have just finished merging #3106 and #3115 into master, which removes our old ant and Java-based build process and replaces it with gulp. Java is no longer required to build Cesium!
For those of you that build Cesium yourself from source, instead of calling ant, you can now use npm run <target> or gulp <target> to kick off build processes. For example
ant build
becomes either
npm run build
or if you prefer gulp directly
gulp build
If you have any questions or run into any problems, let us know. Also check out the Contributor’s Guide for updated instructions. (We also recently added a WebStorm project and instructions for that as well).
For those of you wondering why we used ant to begin with, when Cesium was started there were no other good alternatives and Java/ant was really our best choice. Obviously the landscape has greatly changed in that time and this is the first in a series of changes to modernize Cesium’s build process and improve compatibility with the likes of Node, Browserify, and WebPack.
Great news! Thanks for all your hard work, this is a major step in making Cesium more available to developers on all platforms. Looking forward to taking out all my current browserify shims…
I also have a shim that uses the AMD loader in a development page, so that changes in the source trigger a refresh, and the AMD loader does the rest. I’m using chrome devtools for editing.
With this new gulp build, is there a new ‘recommended workflow’?
TJ, for simply using a Cesium release (i.e. not modifying the Cesium code) we’ve never had a recommended workflow since it’s a completely client side and if you’re just using a release you never need to trigger a rebuild.
For developing and modifying Cesium, we use a node-based server that’s been included with Cesium forever, right now you need to explicitly run it with “node server.js” but with the next release npm start will work. We also have two npm targets
npm run jsHint-watch
npm run build-watch
Which will run in the background and automatically run jsHint or build as needed. Our Eclipse project is also set up to do similar things with no extra tasks in the background.
When I have spare time, I’m thinking about rewriting the server and also incorporating auto-rebuild directly into it, but that won’t be for a while.
So that’s how most of us operate, but we really just encourage people to do what’s works best for them.
Doing ‘ant minify’ used to produce both 'Build/CesiumUnminifiedandBuild/Cesium` outputs.
This was a convenient way to speed up build when you needed both combine (for debug) and minify.
With the new gulp build system, this is no more the case, only the Build/Cesium output directory is created.
Is there a new way to achieve fast minified and unminified builds?
You should be able to run “gulp minify combine” or “npm run combine && npm run minify” to achieve the same behavior. The reason minify no longer depends directly on combine is because it allows them to builds in parallel, which shaves a few minutes off a full release build.
If you are building/deploying your own builds, I would also recommend using minifyRelease instead of just minify, since minifyRelease will remove dev-time error checking and speed up Cesium by 2-4%.
The commands above run sequentially and are equivalent:
gberaudo@wrk29:~/dev/cesium (master)$ ./node_modules/.bin/gulp combine && ./node_modules/.bin/gulp minifyRelease
[10:26:25] Using gulpfile ~/dev/cesium/gulpfile.js
[10:26:25] Starting ‘build’…
[10:26:26] Finished ‘build’ after 370 ms
[10:26:26] Starting ‘generateStubs’…
[10:26:26] Finished ‘generateStubs’ after 50 ms
[10:26:26] Starting ‘combine’…
[10:27:54] Finished ‘combine’ after 1.47 min
[10:28:46] Using gulpfile ~/dev/cesium/gulpfile.js
[10:28:46] Starting ‘build’…
[10:28:46] Finished ‘build’ after 362 ms
[10:28:46] Starting ‘generateStubs’…
[10:28:46] Finished ‘generateStubs’ after 44 ms
[10:28:46] Starting ‘minifyRelease’…
[10:31:07] Finished ‘minifyRelease’ after 2.33 min
gberaudo@wrk29:~/dev/cesium (master)$ ./node_modules/.bin/gulp minifyRelease combine
[10:33:16] Using gulpfile ~/dev/cesium/gulpfile.js
[10:33:16] Starting ‘build’…
[10:33:16] Finished ‘build’ after 354 ms
[10:33:16] Starting ‘generateStubs’…
[10:33:16] Finished ‘generateStubs’ after 45 ms
[10:33:16] Starting ‘minifyRelease’…
[10:33:16] Starting ‘combine’…
[10:37:12] Finished ‘minifyRelease’ after 3.92 min
[10:37:12] Finished ‘combine’ after 3.92 min
So I tried invoking gulp in parallel, but it did not work:
gberaudo@wrk29:~/dev/cesium (master)$ parallel ./node_modules/.bin/gulp ::: combine minifyRelease
[10:53:58] Using gulpfile ~/dev/cesium/gulpfile.js
[10:53:58] Starting ‘build’…
[10:53:59] Finished ‘build’ after 365 ms
[10:53:59] Starting ‘generateStubs’…
[10:53:59] Finished ‘generateStubs’ after 44 ms
[10:53:59] Starting ‘combine’…
[10:53:59] ‘combine’ errored after 559 ms
[10:53:59] Error: Error: Parse error using esprima for file: /home/gberaudo/dev/cesium/Source/Shaders/Builtin/Structs/depthRangeStruct.js
Error: Line 11: Unexpected token ILLEGAL
The content of Source/Shaders/Builtin/Structs/depthRangeStruct.js is garbage, since both gulp processes
They do not run sequentially, they are asynchronous. While gulp (and node in general) is single-threaded, it continues processing when there is blocking io being performed. In your own output you’ll notice that “./node_modules/.bin/gulp combine && ./node_modules/.bin/gulp minifyRelease” took 4 minutes, 42 seconds while “gulp minifyRelease combine” only took 3 minutes and 54 seconds. Also, the old ant process was entirely sequential and only used one core as well (and had the same stomping on files problem that you have already described).
This all being said, there are additional improvements planned for the future and the idea of spawning tasks is something I want to investigate. We also just merged https://github.com/AnalyticalGraphicsInc/cesium/pull/3149, which means for your use case, you can now just run “npm run release” and that will get you the fastest (official) way to produce both combined and minified versions of Cesium.
Thank you Matthew for the clarification and the link to PR 3149.
You are right, my output data shows gulp to be 10% faster when called with both targets.
With the following hack, it is possible touncouple the two targets and trully run them in parallel:
gberaudo@wrk29:~/dev/cesium (master *) git diff
diff --git a/gulpfile.js b/gulpfile.js
index 7806829..3741b73 100644
--- a/gulpfile.js
+++ b/gulpfile.js
@@ -27,6 +27,7 @@ if (/\.0/.test(version)) {
version = version.substring(0, version.length - 2);
}
+var skipGenerateStubs = process.env.SKIP_STUBS
//Gulp doesn’t seem to have a way to get the currently running tasks for setting
//per-task variables. We use the command line argument here to detect which task is being run.
var taskName = process.argv[2];
@@ -287,6 +288,15 @@ gulp.task(‘minifyRelease’, [‘generateStubs’], function() {
gulp.task(‘release’, [‘buildApps’, ‘generateDocumentation’]);
with the next release `npm start` will work. We also have two npm targets
npm run jsHint-watch
npm run build-watch
I have node@v4.4.4 , npm@3.8.8 and Gulp 3.9.1. installed globally (and updated)
Whowever, after '$ npm install cesium' (succesfully, v1.21) I can't make the commands you state here produce the same effect as former server.js (with or without my own gulpfile.js).
In your package.json (dependencies and scripts) I see that you prepared for Gulp, just like I did. Could you please provide a (minimal) gulpfile.js (and dependencies in your package.json) that will do what server.js did? I am eager to test the latest KML specs!
Thanks in advance,
Dieter
----- npm debug log --------
npm ERR! Tell the author that this fails on your system:
npm ERR! node server.js
Tell the author that this fails on your system:
npm ERR! gulp build-watch
It looks like we don’t include the extra node server, apps and testing code in our npm release. If you want to check these out, please download the release from our website: http://cesiumjs.org/downloads.html