Jest coverage reporting

[joaocfernandes]

Bug Report

What is the current behavior?

From what I understood currently by default jest does not perform src coverage extraction for any code that is contained within a /node_modules/. That behavior is determined through the coveragePathIgnorePatterns array of regexp. I have changed it to other value, still coverage extraction is not performed or is erroneous. Please apologize if this was an error on my side.

I have created a sample repository where I try to reproduce some of the sample issues I have came into.
Please refer to the https://github.com/joaocfernandes/jestcov for the following samples.

1. Coverage is not performed, refer to Branch Master

The coverage target file is contained within a src/node_modules/sum.js. The coveragePathIgnorePatterns has an "abcd" regexp. No coverage is extracted. The coverage folder is created at the default location /coverage/.

2. (SOLVED) Coverage is performed but is incorrect, refer to Branch WorkExample

The folder name was renamed from src/node_modules to src/Yeah_modules.
If we run the jest normally the coverage presented reference two different sum.js source files, where which of one does not exists. Global coverage value is incorrect for statements, functions and lines.

3. Coverage is performed but is correct, but file path is incorrect refer to Branch WorkExample

If jest is run ignoring cache the coverage is correctly listed but the src file path is incorrect. It should be listed under src/Yeah_modules/

What is the expected behavior?

Jest should extract coverage taking into account the coveragePathIgnorePatterns , report correct coverage and correct source files path.

WIndows 10
Node v7.8.0
npm 4.2.0
Jest 20.0.4

[joaocfernandes]

Hi,

I also tried to reproduce this on Linux, Ubuntu.

The comparing with the previous 1, and previous 3 they presented thesame behavior than in windows. The 2 presented a different behavior in Linux but still the path of the file is not correct.

node v6.11.0
npm 3.10.10
Jest 20.0.4

[boujeepossum]

i think you should never try to collect coverage from anything inside node_modules. Those files are supposed to be an external dependency that current project doesn't own and their coverage should be collected in their own test suites.

I looked into the issue though. There are two places that prevent files from being instrumented:

1. seems like a jest bug. when i set transformIgnorePatterns config to an empty array (http://facebook.github.io/jest/docs/en/configuration.html#transformignorepatterns-array-string), the defalut value is set to /(?:)/ that matches pretty much any string. and this seems to be the reason why https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/ScriptTransformer.js#L392.

> new RegExp([].join('|'))
/(?:)/

2. second place is inside istanbul itself (the thing that we use to generate coverage) where it forces anything with node_modules in the path to be excluded https://github.com/istanbuljs/istanbuljs/blob/master/packages/test-exclude/index.js#L35-L37

[boujeepossum]

actually i think the first one is not a jest bug, there's a hidden check for transformIgnorePattern.length later in the code https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/ScriptTransformer.js#L402

so i think the only place that forces node_modules to be ignored is in the test-exclude package

[joaocfernandes]

Hello @DmitriiAbramov ,

Thanks for looking into this issue report.
Regarding the 1. it should not impact my created example , because the test description is contained within the test folder, I want Jest to keep the default value for transformIgnorePattern .

My intention is to remove the default value from coveragePathIgnorePatterns , https://facebook.github.io/jest/docs/configuration.html#coveragepathignorepatterns-array-string. I do not have any problem indexing tests, only extracting coverage from files contained /src/node_modules for the reasons mentioned before.

Regarding 2, it can be related with the way the flag coveragePathIgnorePatterns is provided to the test-exclude. I have a similar working setup using mocha+istambul+nyc where I have to pass the option 'include' and 'exclude' the exact same folder negated, as an example:

"nyc": {
    "exclude": ["!**/src/node_modules/**/*.{js,jsx}"],
    "include": [ "**/src/node_modules/**/*.{js,jsx}"],
  }

Still even if I have placed the file on a different folder let's say /src/Yeah_module (which does not require changes to the coveragePathIgnorePatterns config, the coverage extraction works but the path to source file it's not correct on the listing (it should be under src/Yeah_modules/). The covered file sum.js is listed as being on the root.

Thanks,

[stipsan]

Hey @joaocfernandes,

Regarding 2 I think you were affected by a bug that got fixed in this PR: #4432

Can you test if this resolves the rename problem you are experiencing?

[joaocfernandes]

Hi @stipsan,

No duplicates occur now. Still the path to the file is incorrect.
The source file should be under src/Yeah_modules/ .

[stipsan]

Thanks @joaocfernandes.

Since the problem happens when you add --no-cache it makes sense that #4432 didn't fix it. :)
It's an interesting issue though. It displays the previous path but loads the current file contents. Sounds to me like another cache problem 🤔

[jeysal]

2 seems fixed, 1 and 3 are still broken with the current version of Jest.
In case 3, it seems that right now the paths are simply relative to the deepest directory that still contains every file that we have coverage for, but ideally they should be relative to rootDir

[antonovicha]

Try --no-cache jest option.
In our case we refactored code and moved source files around. But jest was picking cached version with pre-refactoring path. Which was not correct any more.

P.S. 2nd or 3rd confusing issue with jest TS cache :(.

[joaocfernandes]

Just made a small check again.
Issue 1. and 3. still occur on Jest 24.9.0 .

joao@joao-ThinkPad-T470p:~/coding/jestcov$ ./node_modules/.bin/jest --version
24.9.0
joao@joao-ThinkPad-T470p:~/coding/jestcov$ ./node_modules/.bin/jest --no-cache
 PASS  __test__/sum.test.js
  ✓ adds 1 + 2 to equal 3 (5ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.028s
Ran all test suites.
joao@joao-ThinkPad-T470p:~/coding/jestcov$ git checkout WorkExample 
Switched to branch 'WorkExample'
Your branch is up to date with 'origin/WorkExample'.
joao@joao-ThinkPad-T470p:~/coding/jestcov$ ./node_modules/.bin/jest --no-cache
 PASS  __test__/sum.test.js
  ✓ adds 1 + 2 to equal 3 (4ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |      100 |      100 |      100 |                   |
 sum.js   |      100 |      100 |      100 |      100 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.205s
Ran all test suites.
joao@joao-ThinkPad-T470p:~/coding/jestcov$ tree src/
src/
└── Yeah_modules
    └── sum.js

1 directory, 1 file

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 7 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.