Instrumenting your webpack-bundled JS code

How to unlock OpenTelemetry observability when using Node.js code bundled with webpack – without giving up on webpack optimizations to your Lambda functions.

OpenTelemetry (OTel) is an emerging industry standard that dev teams use to instrument, generate, collect, and export telemetry to better understand software performance and behavior. At Helios, we promote OTel observability – How? We leverage OTel to provide developers with actionable insights into their code within distributed systems. We give them visibility into how data flows through their applications, enabling them to quickly identify, reproduce and debug issues in their flows.

While OTel is a useful observability framework, there are cases where it doesn’t mesh well with certain tools. In our day-to-day we work with customers who use webpack to bundle their Node.js Lambda functions. This is a common method to reduce Lambda artifact size and optimize cold start times. However, we’ve seen that after OTel is installed, distributed tracing data is partial – instrumentation doesn’t work on the bundled code as expected.

In this blog, I’m going to take you deeper into where the collision between OTel and webpack is. Then I’ll share how we solved the problem at Helios, by auto-starting instrumentations and patching bundled modules explicitly.

Why does OTel clash with webpack? 

When OTel is used to instrument dynamic languages like Javascript there’s a collision with static code bundling tools such as webpack, because of the dynamic patching nature of OTel.

The gist of the problem is that OTel uses the require-in-the-middle module, but webpack switches the require to a __webpack_require. Because OTel cannot find the require it’s looking for, instrumentation is not executed. This issue can escalate even further with tree-shaking of OTel code.

webpack is a static code bundler which transpiles your source code on build time by using static code analysis. When webpack processes your application, it combines every required module that your project uses into one (or more) bundles. Since modules are put inline in the bundle, and required using webpack’s internal require function called __webpack_require, hooking the require on runtime doesn’t work. Also, when it transpiles dynamic imports like import(foo) webpack does not resolve the actual module as foo because foo can be any path to any file in your system or project.

This combination causes OTel instrumentation patching not to work on runtime, and it can potentially be tree-shaken after webpack is unable to detect its dynamic imports.

How to solve the root cause of OTel clashing with serverless webpack

For the purpose of this solution, I’ve used serverless-webpack, but these options can be adapted with minor tweaks to any code implementation. Also, this example specifically refers to AWS, but applies to other cloud providers as well. Below, I go into the solution step-by-step.

Pre-reqs for implementing the solution

• Install serverless-webpack.
  If you’re starting from scratch, you can find detailed info on starting a new serverless-webpack project on the README file.
  npm install serverless-webpack --save-dev
• Create a custom webpack configuration.
  Make sure that modules that are loaded in the Lambda runtime are excluded from your bundle.
  The AWS Lambda execution environment contains a number of libraries such as the AWS SDK for the Node.js runtime (the full list can be found here).

  # serverless.yml
  
  custom:
    webpack:
      webpackConfig: ./webpack.config.js

  // webpack.config.js
  
  module.exports = {
    ..
    externals: [/^aws-sdk$/]
    ..
  }

   

1. Initializing OTel instrumentations on modules that are loaded in the Lambda runtime

Below you can find a basic example for instrumenting Node.js and aws-sdk:

// lambdaOtelInit.js
const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node");
const { registerInstrumentations } = require("@opentelemetry/instrumentation");
const { AwsInstrumentation } = require("@opentelemetry/instrumentation-aws-sdk");

const provider = new NodeTracerProvider();
provider.register()
registerInstrumentations({
  instrumentations: [
    new AwsInstrumentation(),
  ],
});

In order to work properly, OTel instrumentation must be initialized before other libraries are imported into the runtime. You can do this by providing the NODE_OPTIONS configuration to your Lambda:

NODE_OPTIONS="--require lambdaOtelInit.js"

At this point, OTel should be applied properly on the Lambda internal dependencies.

2. Taking care of instrumenting the bundled dependencies properly

There are a couple of different options to accomplish this:

1. Exclude all other instrumented node_modules from the webpack build
   By excluding your instrumented modules from webpack, they will be required regularly at runtime, so instrumentation should work properly.

   // webpack.config.js
   
   module.exports = {
     ..
     externals: [/^aws-sdk$/, /^pg$/] // or you can exclude all modules with nodeExternals()
     ..
   }

   This is the commonly used approach to solve the OTel/webpack clash.
   

2. Auto start instrumentations by patching them explicitly in the bundle
   In the code below, we explicitly require the Node.js Postgres module and patch it, allowing webpack to recognize that the instrumentation patch is being called.
   

   // OTelAutoInit.js
   
   const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
   const { registerInstrumentations } = require('@opentelemetry/instrumentation');
   const { PgInstrumentation } = require('@opentelemetry/instrumentation-pg');
   
   const tracerProvider = new NodeTracerProvider({
     plugins: {
       pg: { path: '@opentelemetry/instrumentation-pg' }
     }
   });
   
   const pgInstrumentation = new PgInstrumentation();
   
   registerInstrumentations({
     tracerProvider,
     instrumentations: [pgInstrumentation]
   });
   
   // this is where the magic happens
   function autoStartPgInstrumentation() {
       try {
           const pg = require('pg')
           pgInstrumentation.patch(pg);
       } catch (error) {
           // error initializing aws-sdk instrumentation;
       }
   }
   
   autoStartPgInstrumentation();
   

   Make sure the init code runs before the handler code. This can be done by adding another entry point prior to the handler entry point.

   // webpack.config.js
   
   module.exports = {
    ..
     entry: ['./OTelAutoInit.js', './handler.js'],
    ..
   };
   

   This approach allows you to benefit from webpack optimizations (including tree-shaking) – while having OTel instrumentation work.

Conclusion

Although it appears that webpack & OTel do not play nice, there are ways to get them to work together and achieve OTel observability. Above I described how developers can ensure their bundled dependencies are instrumented properly by 1/ Initializing OTel instrumentations on modules that are loaded in the Lambda runtime; and 2/ Either (A) Excluding those dependencies from the webpack bundles; OR (B) Including those dependencies in the bundle but explicitly patching them. With this solution, Helios has helped our customers who use webpack benefit from OTel and observability across their application.