Using Dynamic Matrix Config by scbedd · Pull Request #31542 · Azure/azure-sdk-for-js

scbedd · 2024-10-28T00:00:10Z

I've been thinking a lot about how we can abstract away both the package distribution as well as when specific packages have restrictions on what matrix they run.

This PR updates generate-job-matrix.yml with a possible solution. (A lot of the powershell is not fully filled out as this is just a proto)

We will:

Gather all custom matrix configs
Group the packages we need to test (discovered from package properties) by what matrix they are associated with
For each discovered matrix config (starting with a default provided)
- Create the matrix
  - For the packages that belong to the matrix, batch them up and distribute them to the matrix

This way we will be able to generate multiple platform matrixes while also allowing the custom selections from the users.

scbedd · 2024-10-28T00:00:44Z

/azp run js - pullrequest

scbedd · 2024-10-28T00:05:05Z

/azp run js - pullrequest

scbedd · 2024-10-28T02:11:47Z

/azp run js - pullrequest

scbedd · 2024-10-28T02:17:56Z

/azp run js - pullrequest

… and no matter what I do AZDO is refusing to queue the build

scbedd · 2024-10-28T02:40:21Z

/azp run js - matrix-pullrequest

scbedd · 2024-10-28T04:40:34Z

/azp run js - matrix-pullrequest

scbedd · 2024-10-28T17:07:58Z

/azp run js - matrix-pullrequest

benbp · 2024-10-28T20:31:47Z

+                }
+
+                # if we know this is the matrix for our file, we should now see if there is a custom matrix config for the package
+                $matrixConfigList = GetValueSafelyFrom-Yaml $content @("extends", "parameters", "MatrixConfigs")


Does this also handle values behind devops template conditionals? e.g. https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/tests.yml#L33

It does not currently.

benbp · 2024-10-28T20:48:20Z

As I mentioned I was wondering if the generation complexity would be high, but the changes necessary are so small that I think it's very reasonable. I think if you add more examples on inline comments of an inputted matrix set to an outputted matrix set, with the actual package name additions then it will be very approachable to maintain/update going forwards.

scbedd · 2024-10-28T20:59:35Z

As I mentioned I was wondering if the generation complexity would be high, but the changes necessary are so small that I think it's very reasonable. I think if you add more examples on inline comments of an inputted matrix set to an outputted matrix set, with the actual package name additions then it will be very approachable to maintain/update going forwards.

Executing!

…akes 0 sense that it's returning an error. I hate powershell so much

…tually invoking the script

scbedd · 2024-10-29T23:31:54Z

/azp run js - matrix-pullrequest

azure-pipelines · 2024-10-29T23:32:17Z

Azure Pipelines successfully started running 1 pipeline(s).

scbedd · 2024-10-31T00:09:42Z

All the conversations in this PR ended up being commited in Azure/azure-sdk-tools#9281. Closing this PR in favor of #31420 in this repo.

well it's invoking without breaking

4fcfe1d

update the pr trigger

ab67ac5

correct a comment

061f80f

resolve merge conflict

5268d62

disable the protection to only enable on pr. I want to test this crap…

839b444

… and no matter what I do AZDO is refusing to queue the build

add newline

f24289f

scbedd marked this pull request as ready for review October 28, 2024 02:42

scbedd requested review from a team, benbp, ckairen, mikeharder and weshaggard as code owners October 28, 2024 02:42

scbedd added the Do Not Merge label Oct 28, 2024