ESM Bundling for Node with Webpack

It is 2024 and there is a renaissance in the JavaScript world to replace Webpack with other solutions. As much as create-react-app helped Webpack, it was also responsible for this sentiment. Without going further into those details, which is probably a topic for another article, there are two primary reasons for people trying to move away from Webpack:

1. Performance — slow build times.
2. Complex configuration.

However, we should not forget the fact that Webpack is still amazing and nearly works in every possible situation. I have had my hands burned a couple of times when using other bundlers like ESBuilt, Vite, Parcel, Turbopack. I like Vite and have been using it in many frontend projects, especially its explicit and well-documented support for integrating in custom backends, but I still have not been able to use it successfully for backend. I also tried using ESBuild but never the satisfactory output:

How to generate a single file "ESM" bundle using ESBuild for Node.js?

We do not build apps so big to have unacceptably slow build times. There are often reasons for slow build times and which would again be a topic for another day. There is no doubt that configuring Webpack is sometimes a magic art. But, for all its flexibility and support, we have to accept the trade-off.

In this article, we will configure Webpack to generate ESM-only output for Node.js packages that are meant for Node.js and not browsers. This also assumes that you are already familiar with basic concepts of Webpack.

Why bundle for Backend?

Before we jump into the configuration, we have to answer why bundle backend code in the first place. After all, once Node.js reads a file, it stays in memory and it doesn’t have further performance penalty. Node.js natively supports both CommonJS and ESM. Additionally, Node.js has a well-defined module resolution algorithm. Unlike browsers, it knows how to resolve all three types of import specifiers: relative, absolute specifiers as well as bare specifiers (e.g. import from ‘react’).

Again, this could be dedicated post showing all the intricacies of dozens of JavaScript runtimes that exists today but the important points are:

1. Cold starts: Many JS runtimes are serverless. Having a single bundled-file avoids the cold start and boots up fast. Many times, there is restriction on the number of files you can upload on serverless.
2. No npm install: In one of the projects that I worked on, outgoing network requests were not allowed. The only option was to use Yarn v1 offline cache and cache node_modules folders. Both are not pleasurable experiences. And, the existence of bundling (aka compilation for all that we care about) makes it 12-factor compatible. Build exactly once and run it anywhere.
3. Using non-JS resources: Embedding resources is often a problem. In ESM, CJS variables, __filename, __dirname are not readily available. In a proper web app, there is often a need to serve static pages. With webpack loader, HTML files can be imported as a raw string and simply sent.
4. The fractured CJS and ESM world: Packages come in one of the three forms, CJS-only, ESM-only and dual CJS+ESM. But it doesn’t stop there. Some of them are using extensions imports. Some do not have properly configured exports. Many of them still use non-standard module and browser fields in package.json. Once you start de-duping the packages, either with npm dedupe or by using package.json overrides configuration, you will have a hard time figuring out how some CJS package ended up requiring an ESM package instead of importing it. The combinations are mind-boggling. The Webpack takes care of all these issues and yields a single file that will always work for the target it was built-on.
5. Pluggable architecture: Building into a single file makes it easy to create WordPress like plugins and a single file helps us keep the overall architecture trivial.

ESM Criteria

When we are generating ESM output, we will set three core constraints:

1. The produced output should be a single file.
2. The produced output should run on Node.js runtime.
3. The produced output should be ESM-only.

And, finally, our source code could be authored in ESM format using TypeScript or JavaScript. For the simplicity, we will use JavaScript and ESM source in this post.

Setup Basic Skeleton

In order to play with out setup, we will need to setup a minimal skeleton:

1. Empty project with npm init -y.
2. Ensure that package.json has "type": "module" field set.
3. Install required dependencies npm install --save-dev webpack webpack-cli.
4. Create two files index.js and other.js with following content.

///// <ROOT>/src/index.js
export const welcome = 'Welcome';

async function main() {
  console.log('Hello', welcome);
}

main();


///// <ROOT>/src/other.js
export function squareRoot() {
  return Math.sqrt(100);
}

And, finally, we also need webpack.config.js for Webpack configuration with minimal configuration:

///// <ROOT>/webpack.config.js
const config = {
  // Entry point and output
  entry: './src/index.js',
  output: {
    path: process.cwd() + '/dist',
    filename: 'output.js',
  },

  mode: 'production',
};

export default config;

If we compile this now with npx webpack then, it should generate dist/output.js file with following content.

///// <ROOT>/dist/output.js
(() => {
  "use strict";
  !async function () {
    console.log("Hello","Welcome")
  }()
})();

Adding ESM modules using relative specifiers.

The above output is self-explanatory. The generated output is wrapped in IIFE — Immediately Invoked Function Expression. It is neither CJS nor ESM. It is just plain-old ES5 JavaScript code that will run anywhere (Node and browser). Now let’s modify our code and use the squareRoot function exported from other.js module as:

///// <ROOT>/src/index.js
import { squareRoot } from './other.js';

export const welcome = 'Welcome';

async function main() {
  console.log('Hello', welcome);
  console.log('Square root of 100 is', squareRoot());
}

main();

The output is again easy enough to follow:

///// <ROOT>/dist/output.js
(() => {
  "use strict";
  !async function() {
    console.log("Hello","Welcome"),
    console.log("Square root of 100 is", Math.sqrt(100))
  }()
})();

The generated JavaScript output doesn’t contain any reference to CJS or ESM since it doesn’t have any require(), import or import() statements. In fact, Webpack ends up optimizing our code by getting rid of squareRoot function by doing the static analysis of the code as we have set Webpack’s mode to production.

Using Node.js built-in module

Let us try to use Node.js built-in module path by modifying our index.js module:

///// <ROOT>/src/index.js
import path from 'node:path';
import { squareRoot } from './other.js';

export const welcome = 'Welcome';

async function main() {
  console.log('Hello', welcome);

  // Using `relatively` imported function.
  console.log('Square root of 100 is', squareRoot());

  // Using Node.js built-in module.
  console.log('Path', path.join(process.cwd(), 'hello'));
}


main();

If we try to compile this code, then it shall fail as it cannot recognize built-in node:path module:

npx webpack
assets by status 1.67 KiB [cached] 1 asset
orphan modules 58 bytes [orphan] 1 module
./src/index.js + 1 modules 427 bytes [built] [code generated]
node:path 39 bytes [built] [code generated] [1 error]

ERROR in node:path
Module build failed: UnhandledSchemeError: Reading from "node:path" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.

Even if you use plain path module specifier instead of node:path, it would fail. The only change in the error will be:

assets by status 371 bytes [cached] 1 asset
orphan modules 58 bytes [orphan] 1 module
./src/index.js + 1 modules 422 bytes [built] [code generated]

ERROR in ./src/index.js 1:0-24
Module not found: Error: Can't resolve 'path' in '/Users/harshal/code/test-article/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
 - add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
 - install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
 resolve.fallback: { "path": false }
resolve 'path' in '/Users/harshal/code/test-article/src'
  Parsed request is a module
  using description file: /Users/harshal/code/test-article/package.json (relative path: ./src)

The reason why this fails is that path is not a module installed in node_modules and Webpack, primarily being a Frontend bundler, by default, considers web (i.e. browser) as its runtime target. The path module doesn’t make any sense in browser environment.

In order to fix this, we simply need to modify Webpack configuration and add target: 'node' field as follows:

///// <ROOT>/webpack.config.js
const config = {
  // Entry point and output
  entry: './src/index.js',
  output: {
    path: process.cwd() + '/dist',
    filename: 'output.js',
  },

  mode: 'production',

  // NEW FIELD
  target: 'node',
};

export default config;

Addition of target signals Webpack to not bundle Node.js built-in modules like path, fs, child_process, etc. and treat them as runtime/external dependencies. However, if there are other imports or requires, then those would be bundled leaving out only Node.js built-in modules. The generated output (slightly modified for brevity):

///// <ROOT>/dist/output.js
(() => {
  const o = require("node:path");

  !async function(){
    console.log("Hello","Welcome");
    console.log("Square root of 100 is",Math.sqrt(100));
    console.log("Path",o.join(process.cwd(),"hello"));
  }()
})();

This is cool, this should run if we hit node dist/output.js but it won’t as we have specified type: "module" in our package.json.

The presence of "type": "module"field means Node.js runtime will treat any file with *.js extension in <ROOT> and its sub-directory as a ESM and Node.js disallows require() in ESM files.

In order to fix this, either we have to generate ESM output or rename the output.js to output.cjs file. Since, our goal is to actually generate ESM output that can run on Node.js runtime, let’s modify our Webpack configuration to achieve this.

Generating ESM output

Webpack started supporting ESM output for both libraries as well as application bundles quite a while ago but we still need to enable experiment flag for it. We also need to specify node target that’s capable of running ESM modules. At the time of writing this article, Node v20 is already LTS and thus we will use it. So, our final configuration will look like this:

///// <ROOT>/webpack.config.js

const config = {
  // Entry point and output
  entry: './src/index.js',
  output: {
    path: process.cwd() + '/dist',
    filename: 'output.js',

    // NEW FIELD - GENERATE ESM OUTPUT
    module: true,
  },

  mode: 'production',

  // MODIFIED FIELD
  target: 'node20',

  // NEW FIELD - ENABLE ESM FLAG
  experiments: {
    outputModule: true,
  },
};

export default config;

Now if we compile again, we will get following output (again cleaned up a bit for brevity). The output is nice ESM output and it makes use of import statements and also uses module package. The tree shaking has been applied properly as it gets rid of hello variable and replace function call squareRoot with actual value returned by that function:

///// <ROOT>/dist/output.js

import { createRequire } from "module";
const path = createRequire(import.meta.url)("node:path");

// IIFE
!async function() {
  console.log("Hello", "Welcome");
  console.log("Square root of 100 is", Math.sqrt(100));
  console.log("Path", path.join(process.cwd(), "hello"));
}();

This code will also run fine if we simply do node dist/output.js. However, as an astute reader, you will notice that it is still not 100% ESM. Syntactically, it is ESM but it still creates its own version of require() using createRequire function from the module package.

With this approach, try generating the ESM output when your code uses dynamic imports as shown below. I certainly do not wanna go there 😅😅:

async function main() {
  const fs = await import('node:fs/promises');
  const content = await fs.readFile('package.json', 'utf8');

  console.log('Name', JSON.parse(content).name);
}


main();

What if Webpack can actually retain our static as well as dynamic imports as they are (Unlike browsers, Node.js already supports bare import specifiers that we typically use to imported 3rd-party modules from node_modules folder).

Can we actually configure Webpack such that it uses actual import statement and not some custom wrapper that eventually ends up simulating require calls?

The answer to the above question is “Yes”.

Generating “Clean” ESM output

Before we change our configuration one more time, we have to understand why Webpack is doing this for ESM output on Node.js runtime. There are quite a few things to consider:

• Webpack supports not just CJS, ESM but also older module formats.
• Webpack also supports edge-case situations e.g. dynamic imports containing dynamic expressions like import('fs' + '/promises'); etc.
• Further, the presence of node[[X].Y] allows it to rely on module built-in package. ESM and ESM loading are two entirely different things. It is very well possible for have ESM loading completely different. For example, Node.js requires that ESM files have correct file extension while browsers do not need file extension. Browsers rely on <script /> tag attributes as well as Content-Type header to figure this out.

So, what we have to do is to essentially tell Webpack to actually use ESM format as well as standard ESM loading mechanism. We need to change our target from node20 to esX like ES2020, ES2022, etc. But changing the target to something other than node means we have to manually feed Webpack information about what built-in modules should be excluded as those can be assumed to be provided by runtime. The below configuration list all the built-in modules and defines a simple externals function to achieve this. You may consider using existing package like webpack-node-externals:

///// <ROOT>/webpack.config.js

const builtInNodeModules = [
  'assert',      'buffer',
  'cluster',     'crypto',
  'dgram',       'dns',
  'events',      'fs',
  'http',        'http2',
  'https',       'net',
  'os',          'path',
  'process',     'querystring',
  'readline',    'stream',
  'timers',      'tls',
  'tty',         'url',
  'util',        'v8',
  'vm',          'zlib',
  'fs/promises', 'child_process',
  'string_decoder', ,
  'diagnostics_channel',
];


const config = {
  // Entry point and output
  entry: './src/index.js',
  output: {
    path: process.cwd() + '/dist',
    filename: 'output.js',
    module: true,
  },

  mode: 'production',

  // MODIFIED TARGET
  // Do not use Node target.
  // Instead use esX.
  target: 'es2022',

  // Do not use preset like externalsPresets: { node: true },
  // Teach Webpack to handle built-in Node.js modules as external
  async externals({ request }) {
    const isBuiltIn = request.startsWith('node:')
      || builtInNodeModules.includes(request);

    if (isBuiltIn) {
      return Promise.resolve(`module ${request}`);
    }
  },

  experiments: {
    outputModule: true,
  },
};

export default config;

With this configuration, let us try to use dynamic import, static import and some 3rd-party ESM module crypto-random-string:

////// <ROOT>/src/index.js

import path from 'node:path';
import cryptoRandomString from 'crypto-random-string';

async function main() {
  // Using Node.js built-in module.
  console.log('Path', path.join(process.cwd(), 'hello'));

  const fs = await import('node:fs/promises');
  const content = await fs.readFile('package.json', 'utf8');

  // Using Node.js built-in module but dynamic import.
  console.log('Name', JSON.parse(content).name);

  // Using third-party ESM module.
  console.log('Random', cryptoRandomString({ length: 10 }));
}

main();

The code won’t still compile as crypto-random-string uses following exports field in its package.json configuration:

{
  "exports": {
    "types": "./index.d.ts",
    "node": "./index.js",
    "browser": "./browser.js"
  }
}

Since, we remove node as target, and the exports field doesn’t specify import or default field in exports object, we have to adjust Webpack’s resolution a bit by adding following:

{ 
  //... other webpack configuration
  resolve: {
    conditionNames: ['import', 'node'],
  },
}

Now, it should compile and run like a normal Node.js ESM script.

To conclude, you might still say that output with dynamic imports is still not as clean and readable as that of the once produced by other ESM focused bundler. And, that is true. But, we have to keep in mind that Webpack is amazingly flexible and supports multiple target runtimes that no other bundler will support, at least in the near future.

Credits:

• The cover image is shamelessly copied from Freekpik.
• And, modified with little help from Charushila Patil.