Make the action work on all Windows shells

[sondrelg]

There should be a simple way to run an os-matrix, including windows, regardless of the shell you're running. We can detect which shell is running and adapt accordingly.

When we say "all" shells, I guess we actually only care about

• powershell
• bash
• cmd

And cmd might not be that important if we can't easily fix it, but powershell is the default and bash is what we've recommended in the past.

[cortadocodes]

I've had to revert to v1.3.2 of this action to get it working again with Windows.

[sondrelg]

What does your workflow looks like? That's unfortunate.

[cortadocodes]

https://github.com/octue/octue-sdk-python/blob/feature/add-topic-and-subscription-creation-command/.github/workflows/python-ci.yml

[sondrelg]

Can't see that there's anything wrong with the action. Your workflow should work on the latest if you set the shell to bash, as described here I think

These are the tests we're running, and everything seems good. If the action was broken for windows I think we'd see it, no? Can you see anything missing?

[cortadocodes]

I tried setting the shell to bash as recommended:

• It fixed the poetry check step (poetry wasn't found as a commandlet before)
• But the tests in the tox step failed

Here are the workflow runs showing the failures - hope they help!

• Using the latest version of snok/install-poetry
• Using the latest version of snok/install-poetry with the default shell set to bash

[miigotu]

@cortadocodes

You have to install poetry into the tox environment (deps). However, and this might be a silly question, why are you using tox to run the tests when you are using GitHub actions and poetry? poetry run pytest or unittest or whatever is already going to run your tests in a python virtual envelope, and you can separate environments with a GitHub actions matrix.

In any event, there is this also: https://pypi.org/project/tox-poetry/

[cortadocodes]

I can do that if necessary but I didn't have to for any of the previous versions of snok/install-poetry. This is what's making me think there's a bug or a breaking change somewhere in the latest version.

We're still using tox because it allows us to test in other environments than a GHA worker if we need to and it gives us more control than just using GHA. We are using a GHA OS matrix for MacOS, Linux, and Windows. The install-poetry action is only failing on the Windows runner.

[miigotu]

I can do that if necessary but I didn't have to for any of the previous versions of snok/install-poetry. This is what's making me think there's a bug or a breaking change somewhere in the latest version.

We're still using tox because it allows us to test in other environments than a GHA worker if we need to and it gives us more control than just using GHA. We are using a GHA OS matrix for MacOS, Linux, and Windows. The install-poetry action is only failing on the Windows runner.

There for sure is something broken right now. I just figured tox-poetry would be useful. If tox is configured to use an existing environment it could be a problem here, but if tox is creating the environment and then can't find poetry that's because tox is properly sandboxing the environment. Any tools you use inside the toxenv need to be installed by tox itself.

[cortadocodes]

Ah I see, thank you! We're using tox with poetry whitelisted

[Hasenpfote]

EDIT 12/28/2022 yaml file has been modified.

@cortadocodes
When using allowlist_externals, Tox needs a path to find Poetry.

This is one example that works for me.

details

tox.ini

[tox]
envlist =
    py{37,38}
skipsdist = true
isolated_build = true

[testenv]
allowlist_externals =
    poetry
commands_pre =
    poetry install --with test -v
commands =
    poetry run pytest --cov --cov-append --cov-report=term-missing -v
...

.github/workflows/poetry_caching_on_windows_with_install_poetry.yml

name: Poetry caching on Windows with install-poetry

on: workflow_dispatch

permissions:
  contents: read

env:
  config-poetry-version: '' # Empty is the latest version.
  #config-poetry-path: ${USERPROFILE}\.local\bin # An error occurs.
  config-poetry-path: ${USERPROFILE}\.local\venv\Scripts
  config-poetry-cache-paths: |
    ~\.local\VERSION
    #~\.local\bin\poetry.exe # An error occurs.
    ~\.local\venv
  config-poetry-cache-key-fmt: 'd-poetry-{0}-{1}-python-{2}'
  config-venv-cache-key-fmt: 'd-venv-{0}-python-{1}-{2}'

jobs:
  create-cache:
    name: caching 1/2 - ${{ matrix.os }}, ${{ matrix.python-version }}
    strategy:
      fail-fast: false
      matrix:
        os: [windows-latest]
        python-version: ['3.8']

    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up Python ${{ matrix.python-version }}
        id: setup-python
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}

      - name: Load cached Poetry installation
        id: cached-poetry
        uses: actions/cache@v3
        with:
          path: ${{ env.config-poetry-cache-paths }}
          key: ${{ format(env.config-poetry-cache-key-fmt, env.config-poetry-version, matrix.os, steps.setup-python.outputs.python-version) }}

      - name: Install Poetry ${{ env.config-poetry-version }} for Windows
        if: steps.cached-poetry.outputs.cache-hit != 'true'
        env:
          POETRY_VERSION: ${{ env.config-poetry-version }}
        uses: snok/install-poetry@v1

      - name: Add Poetry to the PATH environment variable
        shell: bash
        run: |
          echo "${{ env.config-poetry-path }}" >> $GITHUB_PATH

      - name: Debug
        shell: cmd
        run: |
          set PATH

      - name: Configure Poetry
        run: |
          poetry config virtualenvs.in-project true

      - name: Load cached venv
        id: cached-poetry-dependencies
        uses: actions/cache@v3
        with:
          path: .venv
          key: ${{ format(env.config-venv-cache-key-fmt, matrix.os, steps.setup-python.outputs.python-version, hashFiles('**/poetry.lock')) }}

      - name: Set the environment used by Poetry on Windows 1/2
        if: runner.os == 'Windows' && steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
        id: py
        shell: bash
        run: |
          VER='${{ steps.setup-python.outputs.python-version }}'
          ARR=(${VER//./ })
          echo "tag=${ARR[0]}.${ARR[1]}" >> $GITHUB_OUTPUT

      - name: Set the environment used by Poetry on Windows 2/2
        if: runner.os == 'Windows' && steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
        shell: bash
        env:
          PY_PYTHON: ${{ steps.py.outputs.tag }}
        run: |
          poetry env use py

      - name: Install dependencies
        if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
        run: poetry install --no-interaction --no-root --with dev

      - name: Test with pytest
        run: |
          poetry run tox -e py

  # Same as above.
  load-cache:
    needs: create-cache
    name: caching 2/2 - ${{ matrix.os }}, ${{ matrix.python-version }}

    strategy:
      fail-fast: false
      matrix:
        os: [windows-latest]
        python-version: ['3.8']

    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up Python ${{ matrix.python-version }}
        id: setup-python
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}

      - name: Load cached Poetry installation
        id: cached-poetry
        uses: actions/cache@v3
        with:
          path: ${{ env.config-poetry-cache-paths }}
          key: ${{ format(env.config-poetry-cache-key-fmt, env.config-poetry-version, matrix.os, steps.setup-python.outputs.python-version) }}

      # No need if you have cached Poetry.
      #- name: Install Poetry ${{ env.config-poetry-version }} for Windows
      #  if: steps.cached-poetry.outputs.cache-hit != 'true'
      #  env:
      #    POETRY_VERSION: ${{ env.config-poetry-version }}
      #  uses: snok/install-poetry@v1

      - name: Add Poetry to the PATH environment variable
        shell: bash
        run: |
          echo "${{ env.config-poetry-path }}" >> $GITHUB_PATH

      - name: Configure Poetry
        run: |
          poetry config virtualenvs.in-project true

      - name: Load cached venv
        id: cached-poetry-dependencies
        uses: actions/cache@v3
        with:
          path: .venv
          key: ${{ format(env.config-venv-cache-key-fmt, matrix.os, steps.setup-python.outputs.python-version, hashFiles('**/poetry.lock')) }}

      # No need if you have cached venv.
      #- name: Install dependencies
      #  if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
      #  run: poetry install --no-interaction --no-root --with dev

      - name: Test with pytest
        run: |
          poetry run tox -e py

Tox will fail if the path to Poetry is assigned in POSIX style.

This is an example that works, but it is partially mixed.

Please use Tox3 for verification.
Tox4 doesn't ensure.

LOG: Poetry caching on Windows with install-poetry

[Nicba1010]

Any chance this will get fixed soon? We also have to use 1.3.2

[sondrelg]

PRs are welcome 🙂