Optimising JS bundles for internationalisation

Within Trainline’s SEO focused development team, we have rapidly expanded the content that we deliver into new regions and new languages. With this expansion comes the need to support multiple languages within our codebase as well as the need to deliver just the content the user needs, along with just the JavaScript that is needed for the user.

This blog post will focus specifically on how we optimise our JS bundle for the end user by ensuring we only send them what they need, regardless of how many languages we support.

Our setup

• React for building our UI
• react-intl for handling our language strings
• moment.js for date/time handling and manipulation
• webpack for JS bundling

The complexities of our setup:

• react-intl has locale data which is required to be loaded
• react-intl relies on the browser intl API, however we support older browsers which don’t have the intl API so we need to use the intl polyfill which also has its own locale data
• moment.js has locale data as well
• Each library independently offers advice on how to use public CDN based approaches for loading locale data but for security and reliability reasons we try our best to be self sufficient and self contained. Even if we were to use public CDN approaches, each library would require separate HTTP requests (meaning DNS lookups, handshakes, etc.) to pull down the locale data
• Using the combination of public CDN approach with your own webpack bundles becomes a lot more confusing to deal with in regards to order of execution, exposing global variables, etc.

Day 1 solution — do “as the documentation” says

At the start we were only handling 5 languages so we did as the documentation suggested:

• import all intl and react-intl locale data for the 5 languages
• Use webpack ContextReplacementPlugin to pick out moment.js locale data for all 5 languages

The step-by-step strategy for this approach when ran in the browser is:

// from build time we have already bundled
// all locale data and and translations supported
load react-intl locale data for all languages supported
load moment.js locale data for all languages supported
if (browser intl API is not available) then
  load intl polyfill with all supported languages we desire
end if
render application

However, this means that we are still sending all 5 languages’ locale data (for the three libraries) to the user even though they only need 1, which isn’t viable long term as it will expand the JS bundle as we need to add more languages.

In a larger application, the number of strings that would be sent to the end user increases drastically as well (and then multiplied by languages supported).

As shown from this basic code example when we put it through the webpack bundle analyzer we can see that the main bundle (client.js) includes all the languages bundled in for each library, and intl (vendors~intl.js) does likewise. Also, languages are some of the largest parts of the bundles.

Bundle Analyzer image of basic intl approach

Our solution — webpack code splitting to the rescue!

Webpack code splitting allows for code to be dynamically and conditionally imported depending on what circumstances you need it under. The intl polyfill provides an example of how this works through conditional loading. And though it is considered legacy, we use require.ensure as it does not confuse babel or eslint.

Documentation for require.ensure seems very illusive, so here is some basic documentation for it:

// an array of paths to all files that we will be requiring dynamically for this dynamic bundle
const requiringPaths = [];
// the require parameter is webpack's version of node's require
// even though we specify requiringPaths above
// we still need to require the files individually within the callback
const executionCallback = (require) => {
  // code execution here
};
// bundle-name is optional, but helps with identification
const bundleName = 'bundle-name';
require.ensure(requiringPaths, executionCallback, bundleName); 

The step-by-step strategy for this approach is:

// from build time, we do NOT bundle any locale data or translations
if (browser intl API is not available) then
  load intl polyfill with locale just needed for language
end if
fetch locale data for libraries (react-intl and moment.js) and translations
load locale data for react-intl
load locale data for moment.js
render application

First step is to ignore moment.js locale in your webpack config, this ensures that it does not get put with the main bundle:

• BEFORE: new webpack.ContextReplacementPlugin(/moment[/\\]locale$/, /en-gb|de|es|it|fr/)
• AFTER: new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/)

Second and final step is are the changes in how we tie all this together in entry.js. As the changes are a bit more to show here, you can see the difference between the basic approach and the optimal approach by opening up the files side by side in your browser.

This is a code snippet taken from our new approach demonstrating webpack code splitting:

require.ensure([
  'moment/locale/de.js',
  'react-intl/locale-data/de.js',
  '../translations/de.json'
], (require) => {
  require('moment/locale/de.js');
  localeData = require('react-intl/locale-data/de.js');
  messages = require('../translations/de.json');
  setLocaleData(localeData, messages);
}, 'app-de');

This example shows us loading up the German locale data for moment.js, react-intl, as well as our application translations. We specify the paths to the relevant files that webpack should expect us to require in, require them in, and call a follow-on (callback like) function which sets up moment.js and react-intl with their respective locale datas.

The key takeaway is that dynamic importing allows you to cherry pick exactly what locale data is needed for the end user’s language, even if they lie across multiple libraries.

Of note: there does appear to be a lot of repeated code in entry.js, but we do not want to make the paths dynamic as it would mean that webpack would not know the paths that you are trying to bundle together. Also, GZIP is your friend and the repetition compresses greatly.

As shown from this optimal code example when we put it through the webpack bundle analyzer we can see that the main bundle (client.js) contains only the functional code and no languages bundled in, whilst there are now several separate files that are split out based on the languages supported.

Bundle Analyzer image of optimal intl approach

Comparison by numbers

Sure, the pretty diagrams help, but it is best to know the numbers.

Basic output files and sizes

        Asset          Size         Gzipped
vendors~intl.js       158 KiB       21.1 KiB
        intl.js      79 bytes
      client.js       239 KiB       65.5 KiB

Optimal output files and sizes

Asset                  Size         Gzipped
    app-en.js        8.07 KiB
vendors....js        39.5 KiB       13.3 KiB
   intl-it.js        23.4 KiB       3.91 KiB
   intl-fr.js          24 KiB        4.2 KiB
   intl-es.js        23.9 KiB       3.92 KiB
   intl-de.js        23.8 KiB       4.01 KiB
   intl-en.js        23.6 KiB          1 KiB
    app-it.js         2.7 KiB      606 bytes
    app-fr.js        5.46 KiB      775 bytes
    app-es.js        15.2 KiB      818 bytes
    app-de.js        2.97 KiB      603 bytes
    client.js         197 KiB       62.5 KiB

So from the above two size outputs:

• client.js is our main bundle, which has now reduced ~40KB (~2.5KB Gzipped) based on 5 languages. Note though that this is a super simple application with only 3 strings. As mentioned previously, in a full fledge application there would be hundreds of strings meaning tens to hundreds of kilobytes per language, which is then multiplied by the number of languages supported
• if the consumer does not have the intl support in the browser, it is ~40KB (~13KB Gzipped) for the intl API, and then additional ~24KB (~4KB Gzipped) for the supported single language, verses 40KB + 24KB * N languages (~13KB + ~4KB * N) upfront
• There is overhead in that now we are needing to establish another HTTP request for locale data and messages, but in a large application, especially in SPA applications with many pages, this cost of additional HTTP requests is worth the reduction in size.

Conclusion

Through a better understanding of webpack’s dynamic imports, we can ensure that our end users receive the smallest code bundles needed for them, saving on data, and time to first interaction.

Find the optimal solution on GitHub at https://github.com/davidhouweling/intl-js-bundles/tree/master/intl-optimal-solution