Add live Slack notifications to release workflow (#4046)

* Add live Slack notifications to release workflow

Post a single message to Slack when a release starts, then update it
with threaded replies as each sub-job completes (PyPI publish, release
notes, downstream testing, post-release PR, CLI skill sync). A final
job updates the original message with overall success/failure status
and adds an emoji reaction.

Each thread reply includes relevant links (PyPI history, GitHub
releases, downstream compare URLs, PR links). The Slack announcement
file is uploaded as a thread attachment.

All Slack steps use continue-on-error so failures never block the
release. Notifications are skipped during dry runs.

Requires: SLACK_BOT_TOKEN secret (chat:write, reactions:write,
files:write scopes) and SLACK_CHANNEL_ID repository variable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* emojis

* remove rocket reaction

* Fix slack-complete: add prepare to failure check, add continue-on-error

- Include needs.prepare.result in the failure detection loop. If prepare
  fails after posting the Slack message (e.g. during git push), message_ts
  is set but nothing was published — without this check, the final message
  would incorrectly report success.
- Add continue-on-error: true to match every other Slack step, so a Slack
  API failure doesn't cause the workflow to report as failed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix Slack message newlines: use $'\n' instead of literal \n

In bash double quotes, \n is a literal backslash + n, not a newline.
When passed through jq --arg, Slack receives the two-character string
\n instead of a newline, breaking the block quote rendering.

Use $'\n' concatenation to inject a real newline character.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: Wauplin

.github/workflows/release.yml

@@ -29,7 +29,11 @@
 # - APP_SECRET_PEM_HUGGINGFACE_HUB_BOT   : GitHub App private key for huggingface-hub-bot
 # - APP_ID_HUB_SKILLS_REPO               : GitHub App ID for creating tokens to access the skills repo
 # - APP_SECRET_PREM_HUB_SKILLS_REPO      : GitHub App private key for the skills repo
-# 
+# - SLACK_BOT_TOKEN                      : Slack Bot User OAuth Token (scopes: chat:write, reactions:write, files:write)
+#
+# Required repository variables:
+# - SLACK_CHANNEL_ID                     : Slack channel ID for release notifications
+#
 # An additional secret for OpenCode authentication is needed for automated release notes generation
 # (see the release-notes job below).
 
@@ -72,6 +76,7 @@ jobs:
       is_prerelease: ${{ steps.version.outputs.is_prerelease }}
       since_tag: ${{ steps.version.outputs.since_tag }}
       bump_type: ${{ steps.version.outputs.bump_type }}
+      message_ts: ${{ steps.slack.outputs.message_ts }}
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd  # v6.0.2
         with:
@@ -187,6 +192,41 @@ jobs:
           echo "| Prerelease | \`$IS_PRERELEASE\` |" >> $GITHUB_STEP_SUMMARY
           echo "| Dry run | \`${{ inputs.dry_run }}\` |" >> $GITHUB_STEP_SUMMARY
 
+      - name: Notify Slack — release started
+        if: ${{ !inputs.dry_run }}
+        id: slack
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          VERSION="${{ steps.version.outputs.version }}"
+          TAG="${{ steps.version.outputs.tag }}"
+          BRANCH="${{ steps.version.outputs.branch }}"
+          RELEASE_TYPE="${{ inputs.release_type }}"
+          RUN_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+
+          case "$RELEASE_TYPE" in
+            minor-prerelease) TYPE_EMOJI="🏗️" TYPE_LABEL="Pre-release" ;;
+            minor-release)    TYPE_EMOJI="🚢" TYPE_LABEL="Release" ;;
+            patch-release)    TYPE_EMOJI="🐛" TYPE_LABEL="Patch" ;;
+          esac
+
+          TEXT="${TYPE_EMOJI} *${TYPE_LABEL} \`${TAG}\` started*"$'\n'">Branch: \`${BRANCH}\` • <${RUN_URL}|View run>"
+
+          RESPONSE=$(curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              '{channel: $channel, text: $text, unfurl_links: false}')")
+
+          TS=$(echo "$RESPONSE" | jq -r '.ts // empty')
+          if [[ -n "$TS" ]]; then
+            echo "message_ts=$TS" >> $GITHUB_OUTPUT
+          fi
+
       - name: Create or checkout release branch
         run: |
           BRANCH="${{ steps.version.outputs.branch }}"
@@ -264,6 +304,27 @@ jobs:
         env:
           PYPI_TOKEN: ${{ secrets.PYPI_TOKEN_DIST_HUGGINGFACE_HUB }}
 
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          if [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *PyPI* — \`huggingface-hub ${{ needs.prepare.outputs.version }}\` <https://pypi.org/project/huggingface-hub/#history|published>"
+          else
+            TEXT="❌ *PyPI* — \`huggingface-hub\` publish failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
+
   # ============================================================
   # 3. PUBLISH — PyPI (hf CLI)
   # ============================================================
@@ -289,6 +350,27 @@ jobs:
         env:
           PYPI_TOKEN: ${{ secrets.PYPI_TOKEN_DIST_HF }}
 
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          if [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *PyPI* — \`hf (CLI) ${{ needs.prepare.outputs.version }}\` <https://pypi.org/project/hf/#history|published>"
+          else
+            TEXT="❌ *PyPI* — \`hf (CLI)\` publish failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
+
   # ============================================================
   # 4. RELEASE NOTES — manage GitHub releases
   #
@@ -430,6 +512,28 @@ jobs:
               --generate-notes
           fi
 
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          RELEASES_URL="${{ github.server_url }}/${{ github.repository }}/releases"
+          if [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *Release notes* — <${RELEASES_URL}|draft created> (\`${{ inputs.release_type }}\`)"
+          else
+            TEXT="❌ *Release notes* — failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
+
   # ============================================================
   # 4b. SLACK MESSAGE — generate Slack announcement for prereleases
   # ============================================================
@@ -502,6 +606,55 @@ jobs:
           cat ".release-notes/SLACK_MESSAGE_v${BASE_VERSION}.md" >> $GITHUB_STEP_SUMMARY
           echo '```' >> $GITHUB_STEP_SUMMARY
 
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          VERSION="${{ needs.prepare.outputs.version }}"
+          BASE_VERSION=$(echo "$VERSION" | sed -E 's/\.rc.*//')
+          SLACK_MSG_FILE=".release-notes/SLACK_MESSAGE_v${BASE_VERSION}.md"
+
+          if [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *Slack announcement* — message generated"
+          else
+            TEXT="❌ *Slack announcement* — generation failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
+
+          # Upload the generated Slack announcement as a file in the thread
+          if [[ -f "$SLACK_MSG_FILE" ]]; then
+            FILE_SIZE=$(wc -c < "$SLACK_MSG_FILE")
+            UPLOAD_RESP=$(curl -s -X POST https://slack.com/api/files.getUploadURLExternal \
+              -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+              -H "Content-Type: application/x-www-form-urlencoded" \
+              -d "filename=slack_announcement_v${BASE_VERSION}.md&length=${FILE_SIZE}")
+
+            UPLOAD_URL=$(echo "$UPLOAD_RESP" | jq -r '.upload_url // empty')
+            FILE_ID=$(echo "$UPLOAD_RESP" | jq -r '.file_id // empty')
+
+            if [[ -n "$UPLOAD_URL" && -n "$FILE_ID" ]]; then
+              curl -s -X POST "$UPLOAD_URL" -F "file=@${SLACK_MSG_FILE}" > /dev/null
+              curl -s -X POST https://slack.com/api/files.completeUploadExternal \
+                -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+                -H "Content-Type: application/json" \
+                -d "$(jq -n \
+                  --arg file_id "$FILE_ID" \
+                  --arg channel "$SLACK_CHANNEL_ID" \
+                  --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+                  '{files: [{id: $file_id}], channel_id: $channel, thread_ts: $thread_ts}')" > /dev/null
+            fi
+          fi
+
   # ============================================================
   # 5. DOWNSTREAM TESTING — test RC in transformers, datasets, etc.
   # ============================================================
@@ -593,6 +746,31 @@ jobs:
           echo "- [Actions](https://github.com/huggingface/${{ matrix.target-repo }}/actions)" >> $GITHUB_STEP_SUMMARY
           echo "- [Compare](https://github.com/huggingface/${{ matrix.target-repo }}/compare/main...${BRANCH_NAME})" >> $GITHUB_STEP_SUMMARY
 
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          REPO="${{ matrix.target-repo }}"
+          VERSION="${{ needs.prepare.outputs.version }}"
+          BRANCH_NAME="ci-test-huggingface-hub-${VERSION}-release"
+          COMPARE_URL="https://github.com/huggingface/${REPO}/compare/main...${BRANCH_NAME}"
+          if [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *Downstream* — \`${REPO}\` <${COMPARE_URL}|test branch pushed>"
+          else
+            TEXT="❌ *Downstream* — \`${REPO}\` failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
+
   # ============================================================
   # 6. POST-RELEASE — open PR to bump main to next dev version
   # ============================================================
@@ -621,6 +799,7 @@ jobs:
           git config user.email "huggingface-hub-bot[bot]@users.noreply.github.com"
 
       - name: Create version bump PR
+        id: bump_pr
         env:
           GH_TOKEN: ${{ steps.app_token.outputs.token }}
         run: |
@@ -646,10 +825,35 @@ jobs:
           git commit -m "Post-release: bump version to ${DEV_VERSION}"
           git push -u origin "$PR_BRANCH"
 
-          gh pr create \
+          PR_URL=$(gh pr create \
             --title "Post-release: bump version to ${DEV_VERSION}" \
             --body "Automated version bump after v${VERSION} release." \
-            --base main
+            --base main)
+          echo "pr_url=$PR_URL" >> $GITHUB_OUTPUT
+
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          PR_URL="${{ steps.bump_pr.outputs.pr_url }}"
+          if [[ "${{ job.status }}" == "success" && -n "$PR_URL" ]]; then
+            TEXT="✅ *Post-release* — <${PR_URL}|version bump PR> opened"
+          elif [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *Post-release* — skipped (version already bumped)"
+          else
+            TEXT="❌ *Post-release* — failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
 
   # ============================================================
   # 7. SYNC HF CLI SKILL — update skill docs in skills repo
@@ -724,3 +928,96 @@ jobs:
           labels: |
             automated
             cli-sync
+
+      - name: Notify Slack
+        if: ${{ always() && needs.prepare.outputs.message_ts }}
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          if [[ "${{ job.status }}" == "success" ]]; then
+            TEXT="✅ *CLI skill sync* — done"
+          else
+            TEXT="❌ *CLI skill sync* — failed"
+          fi
+          curl -s -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg thread_ts "${{ needs.prepare.outputs.message_ts }}" \
+              '{channel: $channel, text: $text, thread_ts: $thread_ts, unfurl_links: false}')" > /dev/null
+
+  # ============================================================
+  # 8. SLACK — final status update
+  # ============================================================
+  slack-complete:
+    needs: [prepare, publish-pypi, publish-hf-cli, release-notes, slack-message, test-downstream, post-release, sync-hf-cli-skill]
+    if: ${{ always() && !inputs.dry_run && needs.prepare.outputs.message_ts }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Update Slack message with final status
+        continue-on-error: true
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+        run: |
+          MESSAGE_TS="${{ needs.prepare.outputs.message_ts }}"
+          TAG="${{ needs.prepare.outputs.tag }}"
+          BRANCH="${{ needs.prepare.outputs.branch }}"
+          RELEASE_TYPE="${{ inputs.release_type }}"
+          RUN_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+
+          # Check if any job failed or was cancelled (including prepare itself,
+          # which can fail after posting the Slack message, e.g. during git push)
+          FAILED=false
+          for result in \
+            "${{ needs.prepare.result }}" \
+            "${{ needs.publish-pypi.result }}" \
+            "${{ needs.publish-hf-cli.result }}" \
+            "${{ needs.release-notes.result }}" \
+            "${{ needs.slack-message.result }}" \
+            "${{ needs.test-downstream.result }}" \
+            "${{ needs.post-release.result }}" \
+            "${{ needs.sync-hf-cli-skill.result }}"; do
+            if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
+              FAILED=true
+              break
+            fi
+          done
+
+          case "$RELEASE_TYPE" in
+            minor-prerelease) TYPE_LABEL="Pre-release" ;;
+            minor-release)    TYPE_LABEL="Release" ;;
+            patch-release)    TYPE_LABEL="Patch" ;;
+          esac
+
+          if [[ "$FAILED" == "true" ]]; then
+            TEXT="❌ *${TYPE_LABEL} \`${TAG}\` failed*"$'\n'">Branch: \`${BRANCH}\` • <${RUN_URL}|View run>"
+            REACTION="x"
+          else
+            TEXT="✅ *${TYPE_LABEL} \`${TAG}\` completed*"$'\n'">Branch: \`${BRANCH}\` • <${RUN_URL}|View run>"
+            REACTION="white_check_mark"
+          fi
+
+          # Update the original message
+          curl -s -X POST https://slack.com/api/chat.update \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg text "$TEXT" \
+              --arg ts "$MESSAGE_TS" \
+              '{channel: $channel, text: $text, ts: $ts}')" > /dev/null
+
+          # Add status reaction
+          curl -s -X POST https://slack.com/api/reactions.add \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-Type: application/json" \
+            -d "$(jq -n \
+              --arg channel "$SLACK_CHANNEL_ID" \
+              --arg name "$REACTION" \
+              --arg timestamp "$MESSAGE_TS" \
+              '{channel: $channel, name: $name, timestamp: $timestamp}')" > /dev/null