Skip to content

Consider removing StrykerJS #798

@acelaya

Description

@acelaya

It's sad to be writing this issue, but it might be time to remove StrykerJS from the project, and mutation testing with it, as there are no alternatives.

Context:

First of all, I'm a big fun of mutation testing. I started using it several years ago in PHP, with InfectionPHP, which is actually inspired by StrykerJS.

My first impression was that it was a bit slow, but bearable.

If the project was too big, there was ways to just run it on tests affecting changed code, which was better for CI.

Locally, the relative slowness was easier to handle, and running the whole mutation set was realistic and you could get value from it.

Later I found out about striker, and decided to start using it in JavaScript projects.

However, Stryker has a big problem. It's extremely slow.

I knew the value mutation testing brings, so I decided to use it anyway, only during CI, only over changed code, and as a non-required step that is allowed to fail, and does not block merging pull requests (if the rest is green).

I thought this would give time to the project to improve performance eventually.

They released new versions over time, with performance improvements, but not significant ones.

One of the latest introduced a feature to collect a report that can be used cross-environment to dramatically improve performance, but it requires an initial full project run. In the case of this perfect, the estimated time is... 3 full days... 72 hours.

I haven't been able to get over that initial stage.

As you can see, because of those numbers, I never really use StrykerJS in my dev machine, so I don't really get any real value from it.

I also don't enforce any minimum values or even wait for it to pass in order to merge pull requests.

It's running in the dark, adding no value, and forcing me to keep the dependency up to date.

Lately I even considered migrating from jest to vitest, which StrykerKS does not yet support (they will though), so it's one more reason to remove it from the project entirely.

I'll continue following the project, and if at some point it moves to realistic numbers, I'll consider getting it back.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Done

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions