Deterministic build, Coverlet throws a KeyNotFoundException when ContinuousIntegrationBuild = true.

[chtoucas]

Hello, when I try to run Coverlet (MSBuild or VSTest integration)
with UseSourceLink = true and ContinuousIntegrationBuild = true, it throws a KeyNotFoundException.

Data collector 'XPlat code coverage' message: [coverlet]Coverlet.Collector.Utilities.CoverletDataCollectorException: CoverletCoverageDataCollector: Failed to get coverage result
 ---> System.Collections.Generic.KeyNotFoundException: The given key '' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Coverlet.Core.Coverage.GetSourceLinkUrl(Dictionary`2 sourceLinkDocuments, String document) in /_/src/coverlet.core/Coverage.cs:line 452
   at Coverlet.Core.Coverage.CalculateCoverage() in /_/src/coverlet.core/Coverage.cs:line 350
   at Coverlet.Core.Coverage.GetCoverageResult() in /_/src/coverlet.core/Coverage.cs:line 141
   at Coverlet.Collector.DataCollection.CoverageWrapper.GetCoverageResult(Coverage coverage) in /_/src/coverlet.collector/DataCollection/CoverageWrapper.cs:line 44
   at Coverlet.Collector.DataCollection.CoverageManager.GetCoverageResult() in /_/src/coverlet.collector/DataCollection/CoverageManager.cs:line 93
   --- End of inner exception stack trace ---
   at Coverlet.Collector.DataCollection.CoverageManager.GetCoverageResult() in /_/src/coverlet.collector/DataCollection/CoverageManager.cs:line 98
   at Coverlet.Collector.DataCollection.CoverletCoverageCollector.OnSessionEnd(Object sender, SessionEndEventArgs e) in /_/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs:line 160.

Here is a very simple repository that reproduces the error. The exact commands that I use are:

dotnet test /p:CollectCoverage=true /p:UseSourceLink=true /p:ContinuousIntegrationBuild=true

and

dotnet test --collect:"XPlat Code Coverage" --settings coverlet.USL.runsettings /p:ContinuousIntegrationBuild=true

If I use ContinuousIntegrationBuild = false, everything works fine.

Context:

• .NET Core v3.1.300+
• coverlet.collector v1.3.0 or coverlet.msbuild v2.9.0 with patch for CoverletGetPathMap
• Microsoft.SourceLink.GitHub v1.0.0

[MarcoRossignoli]

Can you confirm that if you remove UseSourceLink = true it works?

[chtoucas]

Actually no. It works when UseSourceLink = true and ContinuousIntegrationBuild = false.
The only case that does not work is when UseSourceLink = true and ContinuousIntegrationBuild = true.

[chtoucas]

Oh sorry, I think I misread your question. Yes it works fine when I remove UserSourceLink = true and whether I set ContinuousIntegrationBuild to true or false.

[MarcoRossignoli]

Yep I mean UserSourceLink = false(or removed) and ContinuousIntegrationBuild = true

[chtoucas]

Just to be crystal clear, the default values for ContinuousIntegrationBuild and UseSourceLink being false, the following commands succeed:

dotnet test /p:CollectCoverage=true
dotnet test /p:CollectCoverage=true /p:ContinuousIntegrationBuild=true
dotnet test /p:CollectCoverage=true /p:UseSourceLink=true

This one fails:

dotnet test /p:CollectCoverage=true /p:UseSourceLink=true /p:ContinuousIntegrationBuild=true

[MarcoRossignoli]

Ok the issue is source link plus deterministic build. Need to investigate further this scenario, thanks for reporting this.

[chtoucas]

Upon further investigations, I think the problem is with generate files.

If we force DeterministicSourcePaths to false, things work again.
The build won't be fully deterministic but Coverlet will work fine when using Source Link on a CI server.

dotnet test /p:CollectCoverage=true /p:UseSourceLink=true /p:ContinuousIntegrationBuild=true /p:DeterministicSourcePaths=false

[MarcoRossignoli]

Thanks a lot for investigation!

[gep13]

@MarcoRossignoli as requested on Twitter here:

https://twitter.com/MarcoRossignoli/status/1280043730960093184

I have updated my build to no longer pass in /p:UseSourceLink=true and I can confirm that the build that was failing, is now succeeding, and coverage reports are now successfully being pushed to codecov.io, where they were failing before.

Thanks for the tip. Let me know if there is anything that I can do to help with the testing of this issue.

[MarcoRossignoli]

Thanks for the test!

[MarcoRossignoli]

Thanks @chtoucas can repro on my local, I'll take a look asap, sorry guys but are very busy months I'm near prod release on my daily job so you know what I mean 🙏
BTW found the issue, fix shouldn't be so hard, confirm that the problem is that deterministic build generates "deterministic" path inside sourcelinks metada, so we need to translate also those.
Dev notes https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Coverage.cs#L356