Fail CI on new linter ecosystem panics (#23597)

Summary
--

The huge number of changes in
#22205 (comment)
should have
obviously been a red flag, but I think it would be nice if CI failed
when new
ecosystem panics were introduced. This PR adds a check for diagnostic
lines that
start with `panic: Panicked at crates/`, raises a `ToolError` if any are
found
in the results from the comparison executable, and then ~~also exits
non-zero if
any errors are returned~~ fails the CI run if the corresponding error
message
was printed.

After trying this out in CI, I opted not to change the script's exit
code itself
because that suppressed the ecosystem comment. It feels a little hackier
this way but preserves the behavior I wanted of both failing CI and
still getting
the ecosystem comment to help with debugging.

Test Plan
--

Local testing on the 0.15.3 tag showing that ruff-ecosystem exited
non-zero and
some manual testing in CI, as you can see below.

Author: ntBre

.github/workflows/ci.yaml

@@ -663,6 +663,12 @@ jobs:
           path: ecosystem-result
           if-no-files-found: "error"
 
+      - name: Fail on ecosystem errors
+        run: |
+          if grep -q "project error" ecosystem-result; then
+            exit 1
+          fi
+
   fuzz-ty:
     name: "Fuzz for new ty panics"
     runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-22.04-16' || 'ubuntu-latest' }}

python/ruff-ecosystem/ruff_ecosystem/check.py

@@ -49,6 +49,8 @@
     r"^(?P<diff>[+-])? ?(?P<location>.*): (?P<code>[A-Z]{1,4}[0-9]{3,4}|[a-z\-]+:)(?P<fixable> \[\*\])? (?P<message>.*)"
 )
 
+PANIC_DIAGNOSTIC_LINE_RE = re.compile(r"^[^:]+: panic: Panicked at ")
+
 CHECK_VIOLATION_FIX_INDICATOR = " [*]"
 
 GITHUB_MAX_COMMENT_LENGTH = 65536  # characters
@@ -530,6 +532,10 @@ async def compare_check(
         comparison_task.result(),
     )
 
+    for line in comparison_output:
+        if PANIC_DIAGNOSTIC_LINE_RE.match(line):
+            raise ToolError(line)
+
     diff = Diff.from_pair(baseline_output, comparison_output)
 
     return Comparison(diff=diff, repo=cloned_repo)