Draco WebAssembly updates break Webpack 4 build

CesiumJS
1

After upgrading Cesium from v1.43 to v1.45, pulling in the glTF Draco compression support, my Webpack build is failing regarding WebAssembly.

After reviewing the PR and issue on GitHub I couldn't really make heads or tails of which way to go here. It sounds like the WebAssembly modules are already built and delivered similarly to the GLSL codes/modules, but possibly there's a build step I need to perform on Cesium now?

Either way, somehow the inclusion of WebAssembly into the mix is tripping up my Webpack build; here are the details (webpack compile):

WARNING in ./node_modules/cesium/Source/ThirdParty/draco_decoder.wasm
Module build failed: TypeError: Cannot read property 'fileLoader' of undefined
    at Object.module.exports (/home/nmschulte/map-project/node_modules/file-loader/index.js:14:28)
@ ./node_modules/cesium/Source/ThirdParty sync ^.\/.*$
@ ./node_modules/cesium/Source/ThirdParty/xss.js
@ ./node_modules/cesium/Source/Core/Credit.js
@ ./node_modules/cesium/Source/Core/IonResource.js
@ ./node_modules/cesium/Source/Core/createWorldTerrain.js
@ ./src/cesium/CesiumGlobe.jsx
@ ./src/App.js
@ ./src/index.js
@ multi ./scripts/config/polyfills.js ./src/index.js

WARNING in ./node_modules/cesium/Source/ThirdParty/Shaders/FXAA3_11.glsl
Module build failed: TypeError: Cannot read property 'fileLoader' of undefined
    at Object.module.exports (/home/nmschulte/map-project/node_modules/file-loader/index.js:14:28)
@ ./node_modules/cesium/Source/ThirdParty sync ^.\/.*$
@ ./node_modules/cesium/Source/ThirdParty/xss.js
@ ./node_modules/cesium/Source/Core/Credit.js
@ ./node_modules/cesium/Source/Core/IonResource.js
@ ./node_modules/cesium/Source/Core/createWorldTerrain.js
@ ./src/cesium/CesiumGlobe.jsx
@ ./src/App.js
@ ./src/index.js
@ multi ./scripts/config/polyfills.js ./src/index.js

ERROR in chunk main [entry]
[name].js
Sync WebAssembly compilation is not yet implemented
Error: Sync WebAssembly compilation is not yet implemented
    at moduleTemplate.hooks.content.tap (/home/nmschulte/map-project/node_modules/webpack/lib/wasm/WasmModuleTemplatePlugin.js:17:13)
    at SyncWaterfallHook.eval [as call] (eval at create (/home/nmschulte/map-project/node_modules/tapable/lib/HookCodeFactory.js:17:12), <anonymous>:7:16)
    at ModuleTemplate.render (/home/nmschulte/map-project/node_modules/webpack/lib/ModuleTemplate.js:49:54)
    at modules.map.module (/home/nmschulte/map-project/node_modules/webpack/lib/Template.js:157:28)
    at Array.map (<anonymous>)
    at Function.renderChunkModules (/home/nmschulte/map-project/node_modules/webpack/lib/Template.js:154:28)
    at compilation.mainTemplate.hooks.modules.tap (/home/nmschulte/map-project/node_modules/webpack/lib/JavascriptModulesPlugin.js:86:23)
    at SyncWaterfallHook.eval [as call] (eval at create (/home/nmschulte/map-project/node_modules/tapable/lib/HookCodeFactory.js:17:12), <anonymous>:7:16)
    at SyncWaterfallHook.lazyCompileHook [as _call] (/home/nmschulte/map-project/node_modules/tapable/lib/Hook.js:35:21)
    at MainTemplate.hooks.render.tap (/home/nmschulte/map-project/node_modules/webpack/lib/MainTemplate.js:128:25)
    at SyncWaterfallHook.eval [as call] (eval at create (/home/nmschulte/map-project/node_modules/tapable/lib/HookCodeFactory.js:17:12), <anonymous>:7:16)
    at SyncWaterfallHook.lazyCompileHook [as _call] (/home/nmschulte/map-project/node_modules/tapable/lib/Hook.js:35:21)
    at MainTemplate.render (/home/nmschulte/map-project/node_modules/webpack/lib/MainTemplate.js:327:34)
    at Object.render (/home/nmschulte/map-project/node_modules/webpack/lib/JavascriptModulesPlugin.js:66:34)
    at Compilation.createChunkAssets (/home/nmschulte/map-project/node_modules/webpack/lib/Compilation.js:1752:29)
    at hooks.optimizeTree.callAsync.err (/home/nmschulte/map-project/node_modules/webpack/lib/Compilation.js:933:10)

(webpack config; paths.cesiumSource === '/home/nmschulte/map-project/node_modules/cesium/Source'):

{
  // ...
  
  resolve: {
    alias: {
      'cesium': paths.cesiumSource,
      // ...
    },
    // ...
  },
  
  plugins: [
    // copy Cesium assets
    new CopyWebpackPlugin([
      { from: path.join(paths.cesiumSource, 'Assets'), to: 'Assets' },
      { from: path.join(paths.cesiumSource, 'ThirdParty'), to: 'ThirdParty' },
      { from: path.join(paths.cesiumSource, 'Widgets'), to: 'Widgets' },
      { from: path.join(paths.cesiumSource, 'Workers'), to: 'Workers' },
      { from: path.join(paths.cesiumSource, '../Build/Cesium/ThirdParty/Workers'), to: 'ThirdParty/Workers', force: true },
      { from: path.join(paths.cesiumSource, '../Build/Cesium/Workers'), to: 'Workers', force: true }
    ]),
    // ...
  ],
  // ...
}

Thanks for looking.

2

It looks like this is a relevant Webpack 4.0 bug:
https://github.com/webpack/webpack/issues/6615

Still, I wonder how Cesium expects this to work with Webpack, and how/if the noted fallback mechanism works.

3

I just ran into a similar issue myself, updating Cesium from 1.40 to 1.45, and using Webpack 3.10. The error message reported “unknown characters” or something similar, which is the usual error when you try to import a non-JS file type that Webpack doesn’t recognize (such as an image without having an image loader in place).

I first attempted to resolve the issue by using wasm-loader, but the resulting Cesium bundle size was all the way up to 5MB (after being about 3MB with 1.40). That’s a huge increase, and way more than I wanted. Since we’re not using any of the 3D tiles stuff, I wound up modifying my Webpack config to ignore any Draco-related content by adding new Webpack.IgnorePlugin(/draco/), which ignored both the WASM file and the similarly-large Draco JS file that are now in the build. That left the generated Cesium bundle at 3.26MB - still an increase over 1.40, but not as bad. Initial app testing with that bundle appears to be working okay.

Out of curiosity, has the Cesium team spent any time thinking about bundle sizes? I realize that Cesium has a ton of functionality, but the library has continued to increase in size over time, and this new Draco stuff is really big. My own team is lucky to be in a situation where JS bundle sizes aren’t a primary concern, but for most people it is.

4

Hey all,

We know the new Draco modules are fairly sizable. Our solution for the time being was that we only load the Draco module in web workers, so it won’t be requested in your application unless you load a Draco encoded model (or 3D tileset), and you could get away with excluding the modules from your bundle if you chose.

If you do need to load a Draco model while using webpack, we do provide a fallback JS module you can load instead by overriding FeatureDetection.supportsWebAssembly to return false.

Overall, we do hope to eventually move to multiple smaller ES6 modules, which would allow developers to reduce the sizes of their bundles much more easily.

Thanks,

Gabby

5

Part of the issue is that I’m generating my own Cesium bundle ahead of time, using Webpack’s “DLL Bundle” feature so that a build of the app references the prebuilt Cesium bundle. Unfortunately, the Draco stuff got pulled into that prebuilt DLL bundle, and it was simplest to just make sure all of that got completely ignored for now.

Long-term, I think moving to ES6 modules should open up some neat possibilities for tree-shaking and cutting down on delivered code size (although I know there’s a lot of cross-importing in Cesium’s codebase, so I’m not sure how much could actually get shaken out if not used by the app).

6

I attempted the same thing with my configuration, but this had no effect
for me.

I will create a test-case/repository to share to see if others can
reproduce. Maybe I'll find something along the way.

Mark, can you share your Webpack configuration? I wonder what your
`resolve` object looks like and any WASM related rules/loaders you may
have. I wonder if the `react-create-app`'s rule/loader structure I've
inherited (with `file-loader` as the fallback/catch-all) is causing my
problem.

7

Our app has a custom Webpack config, rather than something based on Create-React-App.

Here’s the complete config for building a Webpack DLL bundle:

const configValues = require("…/config");

const path = require(“path”);
const webpack = require(“webpack”);
const UglifyJsPlugin = require(“uglifyjs-webpack-plugin”);

const PATHS = configValues.PATHS;

const outputPath = path.join(PATHS.base, “distdll”);

const webpackConfig = {
entry : {
Cesium : [“cesium/Source/Cesium.js”],
},
devtool : “#source-map”,
output : {
path : outputPath,
filename : “[name].dll.js”,
library : “[name]”,
sourcePrefix: “”,
},
plugins : [
new webpack.DllPlugin({
path : path.join(outputPath, “[name]-manifest.json”),
name : “Cesium”,
context : configValues.CESIUM.sourcePath
}),

    // Ignore anything related to Cesium's use of the Draco model compression library,
    // which includes several very large modules.  We don't need any of that.
    new webpack.IgnorePlugin(/draco/),

    new webpack.DefinePlugin({
        'process.env.NODE_ENV': configValues.buildEnv,
    }),

],
module : {
    unknownContextCritical : false,
    unknownContextRegExp: /^.\/.*$/,
    rules : [
        {
            test : /\.css$/,
            use: [
                "style-loader",
                "css-loader"
            ]
        },
        {
            test : /\.(png|gif|jpg|jpeg)$/,
            use : ["file-loader"]
        },
        {
            test: /\.glsl$/,
            use: ['file-loader' ]
        }
    ]        
},

externals: {
    'fs': true,
}

};

if(configValues.isProd) {

webpackConfig.plugins.push(
    new UglifyJsPlugin({
        sourceMap : true,
    })
);

}

module.exports = webpackConfig;

``

8

Also note that I wrote a post last year on the process of setting up a DLL bundle for Cesium: http://blog.isquaredsoftware.com/2017/03/declarative-earth-part-1-cesium-webpack/