Do not include (and execute) uncovered files

[mvorisek]

Q	A
PHPUnit version	9.5.4
PHP version	8.0.5 with Xdebug 3.0.4
Installation Method	Composer

Summary

see https://github.com/atk4/ui/pull/1628/checks?check_run_id=2476898808#step:9:45

phpunit finishes successfully, but when coverage is disabled, it does fail

Current behavior

Generating code coverage report in PHP format ... done [00:00.033]
Class "Behat\MinkExtension\Context\RawMinkContext" not found
Error: Process completed with exit code 2.

How to reproduce

see https://github.com/atk4/ui/pull/1628/checks?check_run_id=2476898808#step:9:45

Expected behavior

When class is not loaded in any phpunit tests, zero coverage must be generated even if parent class is not loadable and no error must be generated.

[sebastianbergmann]

Thank you for your report.

Please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.

Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue.

[mvorisek]

Here is minimal reproducible case:

Let's have file x.php in src/ directory with the following content:

<?php

namespace Pkg;

class BehatContext extends \NotInstalled\BehatMinkContext
{
}

when coverage reporting is enabled, the phpunit currently fails to generate the coverage (empty coverage should be reported).

[sebastianbergmann]

I can reproduce this:

.
├── composer.json
├── phpunit.xml
├── src
│   ├── Bar.php
│   └── Foo.php
└── tests
    └── FooTest.php

phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         cacheResultFile=".phpunit.cache/test-results"
         executionOrder="depends,defects"
         forceCoversAnnotation="true"
         beStrictAboutCoversAnnotation="true"
         beStrictAboutOutputDuringTests="true"
         beStrictAboutTodoAnnotatedTests="true"
         failOnRisky="true"
         failOnWarning="true"
         colors="true"
         verbose="true">
    <testsuites>
        <testsuite name="default">
            <directory suffix="Test.php">tests</directory>
        </testsuite>
    </testsuites>

    <coverage cacheDirectory=".phpunit.cache/code-coverage"
              processUncoveredFiles="true">
        <include>
            <directory suffix=".php">src</directory>
        </include>
    </coverage>
</phpunit>

src/Foo.php

<?php declare(strict_types=1);
final class Foo
{
    public function m(): string
    {
        return 'baz';
    }
}

src/Bar.php

<?php declare(strict_types=1);
final class Bar extends DoesNotExist
{
    public function m(): string
    {
        return 'baz';
    }
}

tests/FooTest.php

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

/**
 * @covers Foo
 */
final class FooTest extends TestCase
{
    public function testOne(): void
    {
        $this->assertSame('baz', (new Foo)->m());
    }
}

$ ./vendor/bin/phpunit --coverage-html /tmp/coverage
PHPUnit 9.5.4 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.0.5 with PCOV 1.0.8
Configuration: /home/sb/bug/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:00.009, Memory: 4.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in HTML format ... Class "DoesNotExist" not found

However, this is expected behaviour in my opinion. src/Bar.php contains "incomplete code" that cannot be executed. And it is not executed, of course. But because processUncoveredFiles="true" is configured in phpunit.xml, PHPUnit tries to process this uncovered (not executed) file: and fails.

Setting processUncoveredFiles="false" works around this problem by instructing PHPUnit to not try to analyse uncovered files.

[mvorisek]

src/Bar.php should be analysed (when processUncoveredFiles="true"), but the analysis should complete even without DoesNotExist class.