Yarn hangs when installing jest@22.0.x

[alexilyaev]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
Yarn hangs on yarn add -D jest@22.0.4.
Works fine with v21.2.1.
I deleted node_modules and yarn.lock and ran yarn cache clean just to make sure.

If the current behavior is a bug, please provide the steps to reproduce and
either a repl.it demo through https://repl.it/languages/jest or a minimal
repository on GitHub that we can yarn install and yarn test.

$ yarn add -D jest@22.0.0
yarn add v1.3.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
⠁
⠁
⠁
[1/1] ⡀ weak
[-/1] ⡀ waiting...
[1/1] ⠁ weak
[1/1] ⡀ weak
[1/1] ⠈ weak
[1/1] ⠁ weak
[-/1] ⠁ waiting...
[-/1] ⠁ waiting...
[-/1] ⠁ waiting...
[-/4] ⠄ waiting...
[-/4] ⠄ waiting...
[3/4] ⠄ weak
[-/4] ⠄ waiting...
[-/4] ⠄ waiting...

With --verbose:

...
verbose 16.456 Removing extraneous file "/Users/alex/.config/yarn/global/node_modules/tabtab/node_modules/.bin".
verbose 16.456 Removing extraneous file "/Users/alex/.config/yarn/global/node_modules/tar-pack/node_modules/.bin".
verbose 16.457 Removing extraneous file "/Users/alex/.config/yarn/global/node_modules/yarn-completions/node_modules".
verbose 16.457 Removing extraneous file "/Users/alex/.config/yarn/global/node_modules/yarn/node_modules/.bin".
verbose 16.458 Removing extraneous file "/Users/alex/.config/yarn/global/node_modules/yarn-completions/node_modules/.bin".
[-/5] ⠄ waiting...
[-/5] ⠄ waiting...
[-/5] ⡀ waiting...
[-/5] ⡀ waiting...
[-/5] ⡀ waiting...
[4/5] ⡀ weak
[4/4] 📃  Building fresh packages...
[1/4] ⠄ spawn-sync
[-/5] ⠁ waiting...
[-/5] ⠁ waiting...
[3/4] ⠠ weak
[-/4] ⠠ waiting...
verbose 17.204 Sun, 24 Dec 2017 14:31:40 GMT tabtab:installer Installing completion script to bashrc directory
Sun, 24 Dec 2017 14:31:40 GMT tabtab:installer Installing completion script to /Users/alex/.bashrc directory
[-/5] ⡀ waiting...
[-/5] ⡀ waiting...
[3/4] ⠁ weak
[-/4] ⠁ waiting...
verbose 18.416 node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@0.6.39
node-pre-gyp info using node@9.3.0 | darwin | x64
node-pre-gyp info check checked for "/Users/alex/.config/yarn/global/node_modules/fsevents/lib/binding/Release/node-v59-darwin-x64/fse.node" (not found)
node-pre-gyp http GET https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.1.3/fse-v1.1.3-node-v59-darwin-x64.tar.gz
node-pre-gyp http 200 https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.1.3/fse-v1.1.3-node-v59-darwin-x64.tar.gz
node-pre-gyp info install unpacking fse.node
[-/5] ⠄ waiting...
[-/4] ⠈ waiting...
[3/4] ⠈ weak
[-/4] ⠈ waiting...
[-/4] ⠈ waiting...

What is the expected behavior?
Should not hang

Please provide your exact Jest configuration and mention your Jest, node,
yarn/npm version and operating system.

Node v9.3.0
Yarn 1.3.2
Mac OS X 10.12.6

[SimenB]

@mjesun Ideas?

[guywaldman]

Experienced this issue as well.

Node v9.3.0
Yarn 1.3.2
MacOS High Sierra 10.13.1

[jnrepo]

npm i w/ jest 22.0.4 throw this error log and fails

> weak@1.0.1 install /Users/me/project/node_modules/weak
> node-gyp rebuild

  CXX(target) Release/obj.target/weakref/src/weakref.o
clang: warning: using sysroot for 'iPhoneSimulator' but targeting 'MacOSX' [-Wincompatible-sysroot]
  SOLINK_MODULE(target) Release/weakref.node
clang: warning: using sysroot for 'iPhoneSimulator' but targeting 'MacOSX' [-Wincompatible-sysroot]
ld: building for OSX, but linking against dylib built for iOS (/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.2.sdk/usr/lib/libc++.tbd). file '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.2.sdk/usr/lib/libc++.tbd' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/weakref.node] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at emitTwo (events.js:125:13)
gyp ERR! stack     at ChildProcess.emit (events.js:213:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Darwin 17.3.0
gyp ERR! command "/usr/local/Cellar/node/8.2.1/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/me/project/node_modules/weak
gyp ERR! node -v v8.2.1
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 
npm WARN react-native@0.51.0 requires a peer of react@16.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN react-native-calendar-picker@5.9.0 requires a peer of babel-preset-expo@^2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN react-native-maps@0.19.0 requires a peer of react@16.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: weak@1.0.1 (node_modules/weak):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: weak@1.0.1 install: `node-gyp rebuild`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

[SamPruden]

Can confirm, I've got another similar case of this with NPM on Windows here. I think I've partially tracked down the issue.

[MY DEV DIRECTORY]\[MY PACKAGE]\node_modules\weak>if not defined npm_config_node_gyp (node "C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  weakref.cc
  win_delay_load_hook.cc
..\src\weakref.cc(17): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory [[MY DEV DIRECTORY]\[MY PACKAGE]\node_modules\weak\build\weakref.vcxproj]
C:\Program Files (x86)\Windows Kits\8.1\Include\um\winnt.h(31): fatal error C1083: Cannot open include file: 'ctype.h': No such file or directory (compiling source file C:\Users\A\AppData\Roaming\npm\node_modu
les\npm\node_modules\node-gyp\src\win_delay_load_hook.cc) [[MY DEV DIRECTORY]\[MY PACKAGE]\node_modules\weak\build\weakref.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:159:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
gyp ERR! System Windows_NT 10.0.10586
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\A\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd [MY DEV DIRECTORY]\[MY PACKAGE]\node_modules\weak
gyp ERR! node -v v9.3.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok

The internal jest-leak-detector module introduces a dependency on weak, which seems to have some problems installing on Windows. @jaylandro seems to have independently tracked this down and filed an issue over there. I've tried to manually npm i weak with the same results, so the problem isn't anywhere in jest itself.

The issue may partly be related to my dev environment, as reports seem a little inconsistent on this. I have the windows-build-tools package and gyp stuff usually works fine, but I don't have much experience messing with gyp stuff. Whether the issue is partly dependent on our environments or not, I think this should be considered a bug as it introduces some new undocumented dependency with no clear fix.

Glancing through the issues on the weak repo it looks like it's not particularly actively maintained and has a few outstanding problems, perhaps if jest is going to take a dependency on this it might make sense for the team to take a fork or something.

I'd appreciate if this dependency could be rolled-back for now until this is sorted, as I just went through the whole process of

Huh that's strange. Oh, Jest can't handle module.parent, I should file a request. Oh, someone just did, and it just got added to the latest version, sweet. npm i --save-dev jest@22. Oh. Damn.

EDIT: I should note that I'm running Windows 10, node v9.3.0, npm v5.6.0, node-gyp v3.6.2, windows-build-tools v1.3.2 and running the npm i from an admin prompt. I haven't tracked down specifically why it's giving the file-not-found error that it is.

[SimenB]

This is really weird, as weak is an optionalDependency. If installing it fails, it should just be ignored... See #4984.

[SamPruden]

@SimenB The copy of jest-leak-detector I'm getting from npm for the jest install has these dependencies, it's listed as both.

  "dependencies": {
    "pretty-format": "^22.0.3",
    "weak": "^1.0.1"
  },
...
  "optionalDependencies": {
    "weak": "^1.0.1"
  },

I'm not too sure what's going on with that package.json though, the version in the repo is significantly shorter than the version that comes down from npm and doesn't contain the duplicate. Is there a build-step doing something to it somewhere?

My full copy of jest-leak-detector/package.json:

{
  "_from": "jest-leak-detector@^22.0.3",
  "_id": "jest-leak-detector@22.0.3",
  "_inBundle": false,
  "_integrity": "sha512-xyVdAmcG8M3jWtVeadDUU6MAHLBrjkP4clz2UtTZ1gpe5bRLk27VjQOpzTwK20MkV/6iZQhSuRVuzHS5kD0HpA==",
  "_location": "/jest-leak-detector",
  "_phantomChildren": {
    "ansi-styles": "3.2.0"
  },
  "_requested": {
    "type": "range",
    "registry": true,
    "raw": "jest-leak-detector@^22.0.3",
    "name": "jest-leak-detector",
    "escapedName": "jest-leak-detector",
    "rawSpec": "^22.0.3",
    "saveSpec": null,
    "fetchSpec": "^22.0.3"
  },
  "_requiredBy": [
    "/jest-runner"
  ],
  "_resolved": "https://registry.npmjs.org/jest-leak-detector/-/jest-leak-detector-22.0.3.tgz",
  "_shasum": "b64904f0e8954a11edb79b0809ff4717fa762d99",
  "_spec": "jest-leak-detector@^22.0.3",
  "_where": "C:\\Users\\A\\Dev\\eq-engine\\node_modules\\jest-runner",
  "bugs": {
    "url": "https://github.com/facebook/jest/issues"
  },
  "bundleDependencies": false,
  "dependencies": {
    "pretty-format": "^22.0.3",
    "weak": "^1.0.1"
  },
  "deprecated": false,
  "description": "Module for verifying whether an object has been garbage collected or not.",
  "homepage": "https://github.com/facebook/jest#readme",
  "license": "MIT",
  "main": "build/index.js",
  "name": "jest-leak-detector",
  "optionalDependencies": {
    "weak": "^1.0.1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/facebook/jest.git"
  },
  "version": "22.0.3"
}

[SimenB]

Oh, that's really interesting! You just found a bug in either lerna or yarn.

$ y info jest-leak-detector dependencies
yarn info v1.3.2
{ 'pretty-format': '^22.0.3',
  weak: '^1.0.1' }
✨  Done in 0.20s.

[SamPruden]

Well that's fun. I've actually never directly touched either of those packages so I can't particularly comment on the details of that. If there's no easy fix for this on your end until that gets patched in lerna/yarn, is it feasible to manually patch the package.json that's going to npm to fix this until that gets sorted?

I'd offer to PR it, but it's probably best I don't mess with a buggy build process that I don't really understand.

[SimenB]

Seems like it's by design... npm/npm#3502

But "If there's a dep in dependencies, but it's also in optionalDependencies, then the optional-ness takes precedence.", so it should still work (just really confusing). I don't understand why it fails your build, then.

If you have a package.json like this:

{
  "name": "some-name",
  "version": "0.0.0",
  "optionalDependencies": {
    "weak": "^1.0.1"
  }
}

and run npm install, does it exit successfully?

[SamPruden]

We were actually thinking along the same lines so I was just about to try that. And that gives the same errors but then skips it when it fails.

C:\Users\A\Dev\temp\node-weak-issue-recreation>npm i

> weak@1.0.1 install C:\Users\A\Dev\temp\node-weak-issue-recreation\node_modules\weak
> node-gyp rebuild


C:\Users\A\Dev\temp\node-weak-issue-recreation\node_modules\weak>if not defined npm_config_node_gyp (node "C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  weakref.cc
  win_delay_load_hook.cc
..\src\weakref.cc(17): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory [C:\Users\A\Dev\temp\node-weak-issue-recrea
tion\node_modules\weak\build\weakref.vcxproj]
C:\Program Files (x86)\Windows Kits\8.1\Include\um\winnt.h(31): fatal error C1083: Cannot open include file: 'ctype.h': No such file or directory (co
mpiling source file C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\src\win_delay_load_hook.cc) [C:\Users\A\Dev\temp\node-weak-
issue-recreation\node_modules\weak\build\weakref.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\A\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:159:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
gyp ERR! System Windows_NT 10.0.10586
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\A\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\A\Dev\temp\node-weak-issue-recreation\node_modules\weak
gyp ERR! node -v v9.3.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN some-name@0.0.0 No description
npm WARN some-name@0.0.0 No repository field.
npm WARN some-name@0.0.0 No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: weak@1.0.1 (node_modules\weak):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: weak@1.0.1 install: `node-gyp rebuild`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

up to date in 4.855s

[SamPruden]

And weirdly, I get the same with this, too...

{
  "name": "some-name",
  "version": "0.0.0",
  "dependencies": {
    "weak": "^1.0.1"
  },
  "optionalDependencies": {
    "weak": "^1.0.1"
  }
}

[SimenB]

and echo $? is 0?

What about npm install jest-leak-detector?

[SamPruden]

So, after running npm i on that test package, the directory now looks like this...

Unless you know what's going on there, that looks like I might have some deeper issue here, I'll try reinstalling npm and see if that fixes anything. The existence of this issue, not opened by me, seems to be testament to the fact that there's some deeper issue here though...

[SamPruden]

echo $? is a unix thing and I'm on Windows, but the equivalent echo %errorlevel% is 0, yes.

npm install jest-leak-detector behaves the same as that demo recreation, shows the error but exits successfully. That seems to work correctly, I'm now unable to recreate the weird files issue, running the test package you suggested now correctly skips the dependency when it fails. This is all really weird.