I am having the exact same issue. Each of the workers is downloaded sequentially, one by one, so the first render of the globe through the Internet gets delayed by 5 seconds at least. I am using the Cesium standard distribution (minified).
Is there a way to improve loading performance or to combine the modules somehow?
As of CesiumJS 1.63, web worker size has reduced significantly after the ES6 migration, which allows us to re-use shared code between workers. You can read more about that in this blog post: https://cesium.com/blog/2019/10/31/cesiumjs-es6/
I believe the reason we cannot combine all workers together is because in order to launch a web worker you must pass a JavaScript file that can run independently.