`Error: EACCES: permission denied, mkdir '/layers/heroku_nodejs-npm-install'` due to NPM writing to `/layers` at runtime

[Crazybun0422]

I reckon these Images are indeed bug existing.

[Crazybun0422]

I tried them all

[joshwlewis]

@Crazybun0422 - Looks like something is trying to create a directory in /layers/heroku_nodejs-npm-install at runtime. I'm not surprised that you would see EACCES for that. Do you have any insights into what might be doing that? Are you calling npm at runtime in some way?

[Crazybun0422]

@Crazybun0422 - Looks like something is trying to create a directory in /layers/heroku_nodejs-npm-install at runtime. I'm not surprised that you would see EACCES for that. Do you have any insights into what might be doing that? Are you calling npm at runtime in some way?

I did npm install when initializing the app, however, this error is reported as I did execute 'node ./web/index.js'. that's difficult to understand why, and this error just appeared when I pull this image last time. so I just guess all these images actually released bug existing.

[Crazybun0422]

I suddenly found this 'heroku/builder-classic:22' is worked, I didn't try this..

[Crazybun0422]

heroku/builder-classic:22 Only this version is work. I tested times of them
@joshwlewis

but seems like all the APIs are deprecated, how I could modify my app to satisy these?

I utilized these images to deploy my shopify app, whether api collections of shopify are not matched for new images of heroku?

[joshwlewis]

@Crazybun0422 heroku/builder-classic:22 is deprecated, and part of the reason is that it uses an older Buildpack API (0.4, 0.10 is current). I don't think you'll be able to fix those deprecation warnings.

I highly suggest heroku/builder:22 instead. I'd like to understand more about the issue with heroku/bulder:22 - can you try starting the app with DEBUG=* environment variable (that should print out a few more logs)?

[edmorley]

@Crazybun0422 Have you had a chance to try the above? Also, could you provide more information on when this stopped working? The build log from the last successful build would be very helpful.

[edmorley]

(I've transferred this issue from https://github.com/heroku/cnb-builder-images to https://github.com/heroku/buildpacks-nodejs)

[edmorley]

xref:
https://community.fly.io/t/re-deploy-my-nodejs-app-and-sadly-got-crashed/18457

[edmorley]

@joshwlewis @colincasey So I can reproduce this easily using the Node.js getting started guide, if I switch to a different user at runtime (that doesn't have write access to /layer). (I actually encountered this issue whilst trying out Heroku-24's split build+run user change against all existing CNBs to see what the fallout would be, and remembered seeing it here.)

It looks like NPM or something is trying to write out a cache to the /layers directory, when it doesn't have permissions to do so. (Only /tmp or $HOME are writable when using a different user at runtime).

For simplicity in this repo I've used the nobody user, but imagine it's instead the run-image-user (which for Heroku-24 would be heroku, vs the build-image user heroku-build):

$ git clone https://github.com/heroku/node-js-getting-started && cd node-js-getting-started
$ pack build node-test
...
$ docker run --rm -it --user nobody --env npm_config_loglevel=silly node-test
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
npm verb cli /layers/heroku_nodejs-engine/dist/bin/node /layers/heroku_nodejs-engine/dist/bin/npm
npm info using npm@10.2.4
npm info using node@v20.11.1
npm verb cache could not create cache: Error: EACCES: permission denied, mkdir '/layers/heroku_nodejs-npm-install'
npm verb logfile could not create logs-dir: Error: EACCES: permission denied, mkdir '/layers/heroku_nodejs-npm-install'
npm verb title npm start
npm verb argv "start"
npm verb logfile logs-max:10 dir:/layers/heroku_nodejs-npm-install/npm_cache/_logs/2024-03-18T10_02_30_443Z-
npm verb logfile could not be created: Error: ENOENT: no such file or directory, open '/layers/heroku_nodejs-npm-install/npm_cache/_logs/2024-03-18T10_02_30_443Z-debug-0.log'
npm verb logfile no logfile created
npm sill logfile done cleaning log files

> node-js-getting-started@0.3.0 start
> node index.js

npm verb stack Error: EACCES: permission denied, mkdir '/layers/heroku_nodejs-npm-install'
npm verb cwd /workspace
npm verb Linux 6.6.16-linuxkit
npm verb node v20.11.1
npm verb npm  v10.2.4
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /layers/heroku_nodejs-npm-install
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/layers/heroku_nodejs-npm-install'
npm ERR!  [Error: EACCES: permission denied, mkdir '/layers/heroku_nodejs-npm-install'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/layers/heroku_nodejs-npm-install'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm verb exit -13
npm verb unfinished npm timer command:start 1710756150561
npm verb unfinished npm timer command:run-script 1710756150568
npm verb code -13

npm ERR! Log files were not written due to an error writing to the directory: /layers/heroku_nodejs-npm-install/npm_cache/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal

[colincasey]

Interesting find @edmorley. I'll play around with this reproduction later today but it seems like the culprit here may be npm's logging.

[edmorley]

I don't think the logging is the root issue, the inability to write a cache is. It's just that when the original error occurs, NPM tries to create a log file about it, which also fails, but a previous error has occurred either way.