Understanding webpack Configuration

Evaluating our webpack is difficult because its behavior depends on its configuration file, and one of the things Rails Webpacker does is generate that configuration file from different inputs. As a result, we can’t see the real webpack configuration in a file.

There’s a partial workaround that will allow us to print out the configuration file, or at least most of it, and it’ll be used to guide our way through the way webpack works, and then we will learn how Webpacker simplifies webpack.

The workaround to let us see our webpack configuration is to go to the file config/webpack/development.js and add the following lines at the end of the script:

Object.defineProperty(RegExp.prototype, "toJSON", {
  value: RegExp.prototype.toString
});

console.log(JSON.stringify(module.exports, null, 2))

At this point, module.exports is an object containing the webpack config as JSON, so we’re simply asking it to print to the console with a two-space indent. A bit of extra code has also been added so that the regular expressions print properly. This won’t get the full-text representation of the webpack config, but it’s as close as we can easily get.

With that code in place, if we run the command bin/webpack, we’ll see the entire webpack configuration in our terminal window.

We’re not going to show the entire thing here in one piece, but we’ll break it down to cover webpack’s main concepts. For each section, we’ll discuss the webpack syntax and what it means, and how Rails Webpacker uses defaults and conventions to generate the configuration.

Mode

All these snippets will be shown one by one, but remember that they are part of the larger configuration file. Also, these snippets may not exactly match the order that our file prints in.

{
  "mode": "development",
  "devtool": "cheap-module-source-map",
}

Unlike Rails, webpack only cares about two modes: development and production

We can set the mode to none, but it is advised not to do that.

This setting sets the NODE_ENV environment variable, and that value might be used within the code to manage features. Setting production is typically used to run some optimization plugins that will not be talked about in detail here.

In Rails, Webpacker sets this variable based on our Rails environment, if we are not in development, we are in production. So, for example, a staging environment gets the production plugins. But Webpacker allows us to maintain separate configuration files based on our Rails environment.

Webpacker allows us to generate, say, a different environment for testing, but the NODE_ENV environment variable in that configuration will still be set to either development or production by the Webpacker config.

The devtool parameter controls the generation of source maps. A source map is a separate file that relates lines in our originally coded file in TypeScript, CoffeeScript, or others with the JavaScript file eventually sent to the browser. With a source map, debugging tools in the browser can show us our original file and not the derived file, making the code easier to navigate.

The cheap-module-source-map is a somewhat faster version of a regular source map that has line-to-line fidelity, but not column-to-column fidelity. In production, Rails Webpacker sets this parameter to source-map, which generates a separate source map for each file and connects it to the webpack pack so that the browser can find the map.

The use of source-map as a production default is recent in Webpacker: originally it didn’t generate production source maps by default. The webpack documentation specifically says, “You should configure your server to disallow access to the Source Map file for normal users!” in a yellow alert box. But David Heinemeier Hansson and the Basecamp team had a change of heart after reflecting on the importance of the openness and availability of HTML and JavaScript in helping people learn the early web, and the default was changed in Rails Webpacker.

If we want to change the behavior back, we can see the options in the webpack documentation.

Entry

The generated configuration file has an entry setting, with one entry for each pack. There are three defined packs here: the application pack that has the Stimulus code and CSS, the venue_display pack that covers the React code, and the hello_react pack that Webpacker generated.

"entry": {
    "application": [
      "/Users/noel/projects/pragmatic/north_by/app/packs/
          entrypoints/application.js",
      "/Users/noel/projects/pragmatic/north_by/app/packs/
          entrypoints/application.scss"
    ],
    "venue_display": "/Users/noel/projects/pragmatic/north_by/app/packs/
        entrypoints/venue_display.tsx"
  },

The entry in a webpack application is a file or files that webpack uses as the starting point for a chunk of code. The expectation is that the entry file will import all the other dependencies in the code (sort of like an asset pipeline manifest file), and webpack will use this information to build a dependency graph. A dependency graph is a list of all the dependencies used by the code so that it can resolve imports and then convert all the code to a browser-friendly file.

By default, a webpack configuration will have one entry point at “main”: "./src/index.js". There are two options for the entry syntax. The application pack, which has two entry files, uses the array syntax. In that syntax, all the entry files are built together.

More commonly, and as our other packs handle it, we can use object syntax, where the key is the name of the pack and the value is the file name that is the entry point for that pack.

We can also specify a function as the value of the entry, which will not be done here, but we might do that if our webpack build depends on a dynamically updated set of files from some external source.

In general, the webpack documents recommend one entry point for each separate HTML page that uses JavaScript components. The entry points can all reference the same underlying code, but by separating a different entry point for each HTML downloaded page, we can focus each page on only the dependencies relevant for that page. This causes a smaller download on each page load to the client at the slight cost of maintaining multiple different pages on the server, where each page would likely have some overlap with the others. This implies that a Single Page App would have only one entry point.

Although it is not used in the configuration, many webpack configurations also have a “context” setting that gives an absolute path that is used as the base path for all the entry points, preventing us from needing to specify the absolute path for each entry point.

For a Rails app, Webpacker uses a convention to prevent our app from spelling out all the entry points explicitly. Webpacker creates an entry point for each file in the application/javascript/packs directory. The Rails app doesn’t need to specify a separate context setting because Webpacker will generate the absolute path for each entry point. Webpacker doesn’t care how we structure the rest of our files, but it does expect every separate pack we want to generate to have a suitable entry file in that directory.

Output

The generated configuration has some things to say about the output of webpack’s run:

"output": {
    "filename": "js/[name]-[contenthash].js",
    "chunkFilename": "js/[name]-[contenthash].chunk.js",
    "hotUpdateChunkFilename": "js/[id]-[hash].hot-update.js",
    "path": "/Users/noel/projects/pragmatic/north_by/public/packs",
    "publicPath": "/packs/"
  }

There’s a lot here, and all of this has to do with where webpack puts the files that it generates.

Let’s start with the path and publicPath options. The path is the absolute internal drive location where we want webpack to place its completed, browser-ready files.

The publicPath option is the flip side of that. It’s the relative location of the output files when requested by the browser. The public path is made part of any URL that webpack needs to create to refer to assets, so it must be consistent with your server configuration, or the browser won’t be able to access any resources.

Webpack uses the filename property to determine the pattern used to name each output pack. In this case, the pattern “js/[name]-[chunkhash].js” indicates that JavaScript files will go into the js subdirectory.

The token [name] will be replaced by the name of the pack, and the [chunkhash] token will be replaced by a hash digest based on the entire content of the pack being outputted. The hotUpdateChunkFilename is similar, except for hot updates. Hot updates and “chunks” are webpack code optimization features and will not be discussed here.

All of these outputs are generated by Rails Webpacker based on defaults we can edit. There are many other options for output, but most of them are not relevant to us from the Rails application. A later part of the configuration generated by Webpacker uses plugins to place generated CSS files in "filename": "css/[name]-[contenthash:8].css".

Loaders

The third main webpack concept is loaders. A loader is a transformation that changes a source file in some way. For example, a Babel loader applies Babel to our code, which is where our TypeScript compiler comes in. Other default loaders handle converting SASS to plain CSS and changing the file name of static file assets. Loaders go into the configuration file as a list inside the module:rules parameter.

The default configuration includes a few loaders. The configuration for the Babel loader will give us the general idea:

{
  "test": "/\\.(js|jsx|mjs|ts|tsx|coffee)?(\\.erb)?$/",
  "include": [
    "/Users/noel/projects/pragmatic/north_by/app/packs"
  ],
  "exclude": "/node_modules/",
    "use": [
      {
        "loader": "/Users/noel/projects/pragmatic/north_by/node_modules/
            babel-loader/lib/index.js",
     "options": {
       "cacheDirectory": true,
       "cacheCompression": false,
       "compact": false
     }
    }
  ]
}

There are two parts here: the test and the use. In this case, the test provides a regular expression. If a file matches that regular expression, the rule for that loader is invoked (other test options are less commonly used). The regular expression here matches any file that ends with .js, jsx, mjs, .ts, .tsx, coffee, and any of that list with the additional suffix of .erb. The erb files can be configured to be parsed by ERB before TypeScript, though currently it is not configured so.

The use option specifies what to do if the file passes the test. In this case, it specifies a loader to pass the file to and a set of options to pass to the loader. Multiple loaders can be specified, in which case they are all applied to the file.

Rails Webpacker includes a loader for static images, PostCSS processing of CSS or SASS, and Babel for processing JavaScript files. Other loaders can be added manually.

Resolve

We’ll see that our webpack config has sections called resolve and resolveLoader. These sections manage how code in our app can address code that is not in the same file.

The resolve section starts with a list of extensions:

"extensions": [
  ".js",
  ".jsx",
  ".mjs",
  ".ts",
  ".tsx",
  ".coffee",
  ".css"
],

This list of extensions has a small but important part to play in module resolution. When we import a file without its extension, as in import * as ActiveStorage from "@rails/activestorage", webpack searches for a file with one of these extensions to determine which file to import. If we want to explicitly include the extension of the file in the import statement, we must have a wildcard ("*") entry in the list.

Another important entry in this part of the configuration is the list of modules:

"modules": [
  "/Users/noel/projects/pragmatic/north_by/app/packs",
  "node_modules"
],

This is the complete list of where webpack should look to find modules, in this case, we look in our app files first, and then in the node_modules directory for third-party modules.

The resolveLoader section is only used to find the webpack’s own loaders. In this case, the configuration is:

"modules": [
  "node_modules"
],
"plugins": [
  {}
]

which means loaders are only searched for in the node_modules directory.

Plugins

Plugins are small bits of code that can have arbitrary effects on the webpack process. Webpack itself splits many of its features into plugins so that they can be added only to projects that use them. A list of official plugins is available online and there are many third-party ones as well.

As for what plugins this application uses, Rails Webpacker adds four of them to the default configuration, but our pretty-printer conventions for the environment fail on plugins because they are typically JavaScript objects in their own right and aren’t converted to JSON usefully. This isn’t exactly what we see in the printout, but it’s what our configuration resolves to:

"plugins": [
  {
    "keys": [ // LOTS OF STUFF ],
    "defaultValues": { // LOTS OF STUFF }
  }
 ]

What do all of these do?

The keys and defaultValues are courtesy of the EnvironmentPlugin, which is provided by webpack and makes our entire list of environment variables visible to webpack in case we want to refer to them in our code.

The CaseSenstivePathsPlugin is a third-party module that prevents conflicts between developers using MacOS and other operating systems. For example, in MacOS, files that have the same name but different cases, like test.js and Test.js, are considered the same file, whereas other systems would consider them different files. Because of this, a developer on a Mac might reference a file using the wrong case. This work for the Mac but produce an error for other machines. This plugin avoids that by causing a build error if an import statement doesn’t match the actual case of the file on disk.

The MiniCssExtractPlugin is the plugin that pulls CSS files that are imported into a separate CSS file on disk. The WebpackAssetsManifest is what generates the hash and chunkhash part of the output file so that saved files have a format like application-8f20d660421d3ae59057.js.

Dev Server

The development configuration has many options for configuring the webpack-dev-server program. We can use webpack-dev-server to live-reload webpack assets while in development. This is a great lead in to Rails Webpacker and how to leverage it during development.