Skip to content

fix(lambda-nodejs): direct esbuild is slower than necessary#37380

Open
rix0rrr wants to merge 6 commits intomainfrom
huijbers/lambda-nodejs-esbuild-speed
Open

fix(lambda-nodejs): direct esbuild is slower than necessary#37380
rix0rrr wants to merge 6 commits intomainfrom
huijbers/lambda-nodejs-esbuild-speed

Conversation

@rix0rrr
Copy link
Copy Markdown
Contributor

@rix0rrr rix0rrr commented Mar 27, 2026

If esbuild is available as a dependency in the current project (so we can skip Docker runs), then esbuild is still being executed via npx or yarn run. The overhead of these can still be quite large if there are many bundles to produce in the course of a CDK application!

  • npx; adds ~400ms
  • yarn run (in the case of Yarn Classic); adds ~150ms (but see the remarks about the dispatcher below)

In both NPM and Yarn Classic's case, we can also run the command directly as node_modules/.bin/esbuild, which adds only a few milliseconds! So we fall back to that solution.

There are still a couple of suboptimal things here that I'm not addressing yet because I don't know how:

  • esbuild is by default a dispatcher script that will execute the right binary for the current platform. A postinstall script normally swaps out the dispatcher script for the actual binary so that dispatch can be skipped, but if it isn't skipped that adds another 100-150ms.
  • Yarn doesn't seem to execute the postinstall, or the postinstall doesn't work in a Yarn environment, meaning that Yarn users always pay the dispatching penalty.
  • Yarn Berry (the modern Yarn) even makes it impossible to run esbuild without going through yarn itself, adding another unskippable 100-150ms for every invocation.

All this means that the fastest local bundling experience will be on (gulp) npm...

(This PR also moves private source files in aws-lambda-nodejs into a private directory, so that it's very clear what parts of the module are free to play with and which ones have backwards compatibility requirements on their API)


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

If `esbuild` is available as a dependency in the current project (so we
can skip Docker runs), then `esbuild` is still being executed via `npx`
or `yarn run`. The overhead of these can be quite large for many small
commands!

- `npx`; adds ~400ms
- `yarn run` (in the case of Yarn Classic); adds ~150ms

In both NPM and Yarn Classic's case, we can also run the command
directly as `node_modules/.bin/esbuild`, which adds only a few
milliseconds! So we fall back to that solution.

There are still a couple of suboptimal things here that I'm not
addressing yet because I don't know how:

- `esbuild` is by default a dispatcher script that will execute the
  right binary for the current platform. A `postinstall` script normally
  swaps out the dispatcher script for the actual binary so that dispatch
  can be skipped, but if it isn't skipped that adds another 100-150ms.
- Yarn doesn't seem to execute the `postinstall`, or the `postinstall`
  doesn't work in a Yarn environment, meaning that Yarn users pay an
  unskippable dispatching penalty.
- Yarn Berry (the modern Yarn) even makes it impossible to run `esbuild`
  without going through `yarn` itself, adding *another* unskippable
  100-150ms for every invocation.

All this means that the fastest local bundling experience will be on
(*gulp*) `npm`...
@aws-cdk-automation aws-cdk-automation requested a review from a team March 27, 2026 13:19
@github-actions github-actions bot added the p2 label Mar 27, 2026
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Mar 27, 2026
@rix0rrr rix0rrr changed the title fix(lambda-nodejs): direct esbuild run is slower than necessary fix(lambda-nodejs): direct esbuild is slower than necessary Mar 27, 2026
Copy link
Copy Markdown
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@rix0rrr rix0rrr added pr-linter/exempt-test The PR linter will not require test changes pr-linter/exempt-integ-test The PR linter will not require integ test changes labels Mar 27, 2026
@aws-cdk-automation aws-cdk-automation dismissed their stale review March 27, 2026 13:21

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution/core This is a PR that came from AWS. p2 pr/needs-maintainer-review This PR needs a review from a Core Team Member pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/exempt-test The PR linter will not require test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants