openmpi not showing up in migrations

[minrk]

Something funky is going on in migrators.

openmpi uses both ucx and libhwloc, but both ucx120 and libhwloc2122 migrations completed without issuing PRs to openmpi.

This has led to downstream failures like:

• Rebuild for libhwloc 2.12.2 conda-forge/petsc-feedstock#274
• Rebuild for ucx 1.20 conda-forge/adios2-feedstock#129

I'm not sure how the graph is computed, but openmpi is a v1 recipe uses these dependencies conditionally:

      host:
        - if: mpi_type != 'external'
          then:
            - libevent
            - libfabric-devel ${{ libfabric }}.*
            - libhwloc
            - libpmix-devel
            - zlib
            - if: linux
              then:
                - libnl
            - if: linux and not ppc64le
              then:
                - ucc
                - ucx
            - if: with_cuda
              then:
                - cuda-version ${{ cuda_compiler_version }}.*

Possibly related: are migrations supposed to be completed when all PRs have been opened, or when they are merged? Because both migrators have now marked as completed with several open, failing PRs.

[beckermr]

This does appear to be a bug. The best way to proceed is to add a test and see what is happening.

Migrations get closed in a few ways, but generally after all PRs are issued, it is safe to close them. We have deadman switches of sorts to prevent stragglers etc.

[minrk]

poking around in the graph, openmpi has this parse error caused by the compiler stub regex, which I'm guessing is the cause:

feedstock parsing error: Error running 'conda-forge-tick-container parse-feedstock --existing-feedstock-node-attrs - --log-level info --mark-not-archived' in container - error CalledProcessError raised:
1, ['rattler-build', 'build', '--render-only', '--variant-config', '/tmp/tmpmi63cx6o/openmpi-feedstock-main/.ci_support/linux_64_channel_targetsconda-forge_maincuda_compiler_version12.9mpi_typeconda.yaml'], '', '
 ╭─ Finding outputs from recipe
 │ Loading variant config file: "/tmp/tmpmi63cx6o/openmpi-feedstock-main/.ci_support/linux_64_channel_targetsconda-forge_maincuda_compiler_version12.9mpi_typeco
 │ nda.yaml"
 │
 ╰─────────────────── (took 0 seconds)
Error:   × Failed to parse recipe

Error: 
  × failed to parse match spec: invalid package name 'c')': 'c')' is not a
  │ valid package name. Package names can only contain 0-9, a-z, A-Z, -, _,
  │ or .
     ╭─[../.tmpU2CK1v/recipe.yaml:128:11]
 127 │         - ${{ pin_subpackage('openmpi', exact=True) }}
 128 │         - c') | replace(' =', '>=_compiler_stub
     ·           ──────────────────┬──────────────────
     ·                             ╰── error parsing `c') | replace(' =', '>=_compiler_stub` as a match spec
 129 │     tests:

pointing to utils.replace_compiler_with_stub having an imprecise regex, because:

replace_compiler_with_stub("${{ compiler('c') | replace(' =', '>=') }}"

gives

"c') | replace(' =', '>=_compiler_stub"

I don't know enough of what the compiler stub is for to work around this. The most consistently correct would probably be to replace just compiler('c') with 'c_compiler_stub', leaving the surrounding ${{ ... }}, but I don't know if that solves whatever problem is solved by the stub in the first place.

[beckermr]

We should fix the regex. The stubs are used to let the bot not worry about tracking compiler dependencies in its graph.

[minrk]

Given the input:

${{ compiler('c') | replace(' =', '>=') }}

should the output be:

c_compiler_stub

or

${{ 'c_compiler_stub' | replace(' =', '>=') }}

or leave it unmodified as just not one of the things being replaced:

${{ compiler('c') | replace(' =', '>=') }}

I can do any of the above, but I'm not sure which would be best. Just replacing compiler('c') with 'c_compiler_stub' seems best to me.

[beckermr]

The output should be c_compiler_stub. See eg https://github.com/regro/cf-graph-countyfair/blob/cb40b2ccb93dce2d88e7025c7bab6e36e1dc7007/node_attrs/6/f/d/3/0/galsim.json#L56

[minrk]

ok, so any ${{ }} template that contains any reference to a compiler should be replaced with {c}_compiler_stub, no matter what else might be in there?

I started with #5609 because it seems to me to be the most likely to be correct if there's anything else there, so it's producing ${{ 'c_compiler_stub' }}. The result of the rendered json in both cases remains the same c_compiler_stub.

[beckermr]

Yes. Replace the whole thing.