ci: migrate e2e tests from EKS/Kubernetes to GitHub runners#927
Merged
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR migrates E2E testing infrastructure from Kubernetes manifests to TOML-based scenario configurations and GitHub runner execution, while adding rustls TLS crypto provider initialization for the U5C feature. Eight Kubernetes YAML manifests are removed and replaced with fourteen TOML scenarios and three helper scripts. The workflow shifts from EKS-based orchestration to runner-based matrix execution. ChangesE2E Infrastructure Migration
U5C TLS Crypto Provider Initialization
🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (4)
src/filters/work_stats.rs (3)
64-66: ⚡ Quick winCloning ChainEvent on every message may cause performance overhead.
Every event is cloned before being forwarded downstream. Since
ChainEvent::Recordcan contain large payloads likeCborBlockorParsedBlock, this clone operation may become a bottleneck in high-throughput scenarios.Consider measuring the performance impact in production. If cloning proves expensive, a potential optimization would be to refactor the filter to operate on
gasket::messaging::Message<ChainEvent>directly (forwarding the message itself rather than extracting and re-wrapping the payload), though this may require changes to the gasket worker pattern.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/filters/work_stats.rs` around lines 64 - 66, The current execute method clones every ChainEvent before forwarding (stage.output.send(unit.clone().into()).await...), which is costly for ChainEvent::Record payloads like CborBlock/ParsedBlock; change the filter to forward the existing gasket::messaging::Message<ChainEvent> (or otherwise borrow/move the payload) instead of cloning: update the worker signature or wiring so execute receives and forwards the Message<ChainEvent> directly (or uses into_inner/move semantics) and replace the unit.clone().into() call with a send of the original message, ensuring no extra copies of large payloads are made.
97-126: ⚡ Quick winConsider adding unit tests for WorkStats filter logic.
The WorkStats filter contains non-trivial stateful logic—tracking block counts, enforcing finalization policy, and coordinating pipeline shutdown—but has no dedicated unit tests. While
framework/mod.rstests theshould_finalizefunction, the filter's integration of metrics, state management, and the finished-flag lifecycle remains untested.Consider adding tests that:
- Verify block counting increments only on Apply events
- Confirm the filter forwards events unchanged
- Validate that
WorkSchedule::Doneis returned after finalization- Test that Undo/Reset events don't increment block_count but do participate in slot-based finalization
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/filters/work_stats.rs` around lines 97 - 126, Add unit tests for the WorkStats filter to cover its stateful behaviors: use Config::bootstrapper to construct a Stage (or directly instantiate Stage/FinalizeConfig with fields until_hash, max_block_slot, max_block_quantity) then simulate sequences of events to assert that (1) block_count only increments on Apply events, (2) events are forwarded unchanged by the filter, (3) after the finalization conditions the filter returns WorkSchedule::Done, and (4) Undo/Reset events do not increment block_count but still participate in slot-based finalization checks; create test cases that drive Stage/WorkStats through Apply/Undo/Reset and slot changes and assert the expected block_count, forwarded outputs, and finalization result.
71-80: ⚡ Quick winObservability gap:
latest_slotgauge excludes Undo/Reset events.The
latest_slotgauge is only updated when processingApplyevents (line 75), but the finalization policy at line 84 evaluates the point from all event types includingUndoandReset. This inconsistency means the gauge won't reflect the actual "latest point" that the finalization logic considers, reducing observability during rollbacks or resets.📊 Proposed fix to update gauge for all event types
let point = match unit { ChainEvent::Apply(point, _) => { self.blocks += 1; stage.block_count.inc(1); - stage.latest_slot.set(point.slot_or_default() as i64); point.clone() } ChainEvent::Undo(point, _) => point.clone(), ChainEvent::Reset(point) => point.clone(), }; + stage.latest_slot.set(point.slot_or_default() as i64); + stage.ops_count.inc(1);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/filters/work_stats.rs` around lines 71 - 80, The latest_slot gauge is only updated in the ChainEvent::Apply branch but the variable point (used later by the finalization policy) can come from Apply, Undo, or Reset; update stage.latest_slot for all branches so the gauge reflects the actual point the finalization logic uses—i.e., after you set point in the match over ChainEvent (variants ChainEvent::Apply, ChainEvent::Undo, ChainEvent::Reset), call stage.latest_slot.set(point.slot_or_default() as i64) in each branch (keep self.blocks increment and stage.block_count.inc(1) only in the ChainEvent::Apply branch)..github/e2e/configs/aws-lambda-10-blocks.toml (1)
22-25: ⚡ Quick winConsider parameterizing the AWS account ID in the Lambda ARN.
The Lambda function ARN contains a hardcoded AWS account ID (
295114534192). While this may be acceptable for E2E test configurations, consider using an environment variable or GitHub Actions secret to avoid hardcoding infrastructure details in version control.This improves maintainability if the test account changes and reduces exposure of account identifiers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/e2e/configs/aws-lambda-10-blocks.toml around lines 22 - 25, The Lambda ARN in the [sink] block for type "AwsLambda" hardcodes the AWS account ID (295114534192) in the function_name value; replace that hardcoded account ID with a parameter sourced from an environment variable or CI secret (e.g., interpolate a variable like ${AWS_ACCOUNT_ID} or use a GitHub Actions secret) so the function_name ARN is constructed at runtime instead of committed with the literal account number.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/e2e.yaml:
- Around line 115-119: The socket-wait loop around the path
"${RUNNER_TEMP}/sockets/node0.socket" can exit after retries without the socket
actually existing, delaying failure until later; after the for loop that checks
-S and sleeps (the loop using seq 1 30), add a post-loop test that fails the job
immediately if the socket still doesn't exist (e.g., check [ -S
"${RUNNER_TEMP}/sockets/node0.socket" ] and exit 1 with a clear error message)
so the broken tunnel setup errors fast rather than consuming downstream
timeouts.
- Around line 70-72: The job-level continue-on-error: ${{ matrix.kind == 'n2c'
}} hides all n2c failures; remove that job-level setting and instead apply
continue-on-error only to the specific step that performs the Demeter `dmtrcli`
port-forward/bootstrap (e.g., the step named like "start dmtrcli port-forward"
or "wait for dmtrcli"); update that step to include continue-on-error: true and
keep the job failing for any other step errors so real regressions (image pulls,
daemon panics, test assertions) still mark the workflow red.
- Around line 17-23: CI workflow uses floating action tags and runs a remote
install script insecurely; update every `uses:` entry (e.g.,
actions/checkout@v4, docker/setup-buildx-action@v3, docker/login-action@v3,
docker/build-push-action@v6, aws-actions/configure-aws-credentials@v4) to pinned
commit SHAs, replace the `curl .../demeter-run/cli/.../install.sh | sh`
invocation with a download of a specific release/commit and verify via
checksum/signature before executing, change `continue-on-error: ${{ matrix.kind
== 'n2c' }}` from job-level to a narrowly scoped step-level flag or remove it so
failures bubble unless only the specific tunnel/socket step is expected to be
flaky, and after the `seq 1 30` loop that waits for
${RUNNER_TEMP}/sockets/node0.socket add an explicit existence check that fails
the job immediately with a clear error if the socket file was not created (to
avoid burning full timeout in subsequent `timeout 1800 docker run ...`).
- Around line 108-113: Replace the unsafe pipe to shell invocation "curl -fsSL
https://raw.githubusercontent.com/demeter-run/cli/main/install.sh | sh" with a
pinned download + integrity check: fetch a specific release/tag/commit URL (or
its tarball) into a temp file, verify its SHA256 (or GPG) checksum, and only
then run it (or use a packaged release binary); update the workflow step that
currently runs that curl|sh line to download to
"${RUNNER_TEMP}/demeter_install.sh", verify the checksum against a stored value
(or fetch a signed checksum and verify signature), and exec the vetted file
instead of piping directly to sh.
---
Nitpick comments:
In @.github/e2e/configs/aws-lambda-10-blocks.toml:
- Around line 22-25: The Lambda ARN in the [sink] block for type "AwsLambda"
hardcodes the AWS account ID (295114534192) in the function_name value; replace
that hardcoded account ID with a parameter sourced from an environment variable
or CI secret (e.g., interpolate a variable like ${AWS_ACCOUNT_ID} or use a
GitHub Actions secret) so the function_name ARN is constructed at runtime
instead of committed with the literal account number.
In `@src/filters/work_stats.rs`:
- Around line 64-66: The current execute method clones every ChainEvent before
forwarding (stage.output.send(unit.clone().into()).await...), which is costly
for ChainEvent::Record payloads like CborBlock/ParsedBlock; change the filter to
forward the existing gasket::messaging::Message<ChainEvent> (or otherwise
borrow/move the payload) instead of cloning: update the worker signature or
wiring so execute receives and forwards the Message<ChainEvent> directly (or
uses into_inner/move semantics) and replace the unit.clone().into() call with a
send of the original message, ensuring no extra copies of large payloads are
made.
- Around line 97-126: Add unit tests for the WorkStats filter to cover its
stateful behaviors: use Config::bootstrapper to construct a Stage (or directly
instantiate Stage/FinalizeConfig with fields until_hash, max_block_slot,
max_block_quantity) then simulate sequences of events to assert that (1)
block_count only increments on Apply events, (2) events are forwarded unchanged
by the filter, (3) after the finalization conditions the filter returns
WorkSchedule::Done, and (4) Undo/Reset events do not increment block_count but
still participate in slot-based finalization checks; create test cases that
drive Stage/WorkStats through Apply/Undo/Reset and slot changes and assert the
expected block_count, forwarded outputs, and finalization result.
- Around line 71-80: The latest_slot gauge is only updated in the
ChainEvent::Apply branch but the variable point (used later by the finalization
policy) can come from Apply, Undo, or Reset; update stage.latest_slot for all
branches so the gauge reflects the actual point the finalization logic
uses—i.e., after you set point in the match over ChainEvent (variants
ChainEvent::Apply, ChainEvent::Undo, ChainEvent::Reset), call
stage.latest_slot.set(point.slot_or_default() as i64) in each branch (keep
self.blocks increment and stage.block_count.inc(1) only in the ChainEvent::Apply
branch).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b0ac7801-b0ad-4255-8170-28d746ea7dac
📒 Files selected for processing (24)
.github/e2e/assert-n2c-preview-origin-blocks.yaml.github/e2e/assert-n2c-preview-tip-blocks.yaml.github/e2e/assert-n2n-mainnet-origin-blocks.yaml.github/e2e/assert-n2n-mainnet-tip-blocks.yaml.github/e2e/assert-n2n-preview-origin-blocks.yaml.github/e2e/assert-n2n-preview-tip-blocks.yaml.github/e2e/aws-lambda-10-blocks.yaml.github/e2e/aws-s3-500-blocks.yaml.github/e2e/aws-sqs-10-blocks.yaml.github/e2e/configs/assert-n2c-preview-origin-blocks.toml.github/e2e/configs/assert-n2c-preview-tip-blocks.toml.github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml.github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml.github/e2e/configs/assert-n2n-preview-origin-blocks.toml.github/e2e/configs/assert-n2n-preview-tip-blocks.toml.github/e2e/configs/aws-lambda-10-blocks.toml.github/e2e/configs/aws-s3-500-blocks.toml.github/e2e/configs/aws-sqs-10-blocks.toml.github/e2e/gcp-pubsub-10-blocks.yaml.github/e2e/serviceaccount.yaml.github/workflows/e2e.yamlsrc/filters/mod.rssrc/filters/work_stats.rssrc/framework/mod.rs
💤 Files with no reviewable changes (11)
- .github/e2e/assert-n2n-mainnet-origin-blocks.yaml
- .github/e2e/assert-n2n-preview-tip-blocks.yaml
- .github/e2e/gcp-pubsub-10-blocks.yaml
- .github/e2e/aws-s3-500-blocks.yaml
- .github/e2e/aws-lambda-10-blocks.yaml
- .github/e2e/aws-sqs-10-blocks.yaml
- .github/e2e/assert-n2n-preview-origin-blocks.yaml
- .github/e2e/assert-n2c-preview-origin-blocks.yaml
- .github/e2e/serviceaccount.yaml
- .github/e2e/assert-n2n-mainnet-tip-blocks.yaml
- .github/e2e/assert-n2c-preview-tip-blocks.yaml
Comment on lines
17
to
23
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Login to DockerHub | ||
| - name: Login to GHCR | ||
| uses: docker/login-action@v3 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the referenced workflow with line numbers around the build/test jobs.
echo "=== .github/workflows/e2e.yaml (lines 1-220) ==="
nl -ba .github/workflows/e2e.yaml | sed -n '1,220p'
echo "=== Search for unpinned uses: (actions/*) and specific actions mentioned ==="
rg -n "uses:\s*(actions/checkout|docker/setup-buildx-action|docker/login-action|aws-actions/configure-aws-credentials|aws-actions/|docker/)[`@a-zA-Z0-9_./-`]*" .github/workflows/e2e.yaml
echo "=== Search for remote script execution patterns (curl | sh, bash -c, pipe to sh) ==="
rg -n "(curl|wget)\s+.*\|\s*(sh|bash)" .github/workflows/e2e.yaml
rg -n "raw\.githubusercontent\.com.*\|\s*(sh|bash)" .github/workflows/e2e.yaml
rg -n "run:\s*.*\|\s*(sh|bash)" .github/workflows/e2e.yaml
echo "=== Check continue-on-error usage and where it is applied ==="
rg -n "continue-on-error" .github/workflows/e2e.yaml
echo "=== Inspect any socket wait loop / timeout handling ==="
rg -n "(socket|wait|nc -z|nc |/dev/tcp|localhost|timeout|sleep)" .github/workflows/e2e.yamlRepository: txpipe/oura
Length of output: 165
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Repo tree for workflow files ==="
ls -R .github/workflows || true
echo "=== Show .github/workflows/e2e.yaml (with line numbers) ==="
awk '{print NR "\t" $0}' .github/workflows/e2e.yaml | sed -n '1,260p'
echo "=== Search for unpinned uses: (actions/* and docker/aws actions) ==="
rg -n "uses:\s*(actions/checkout|docker/setup-buildx-action|docker/login-action|aws-actions/configure-aws-credentials|aws-actions/|docker/)[`@a-zA-Z0-9_./-`]*" .github/workflows/e2e.yaml || true
echo "=== Search for remote script execution patterns (curl|wget piped to sh/bash) ==="
rg -n "(curl|wget)\s+.*\|\s*(sh|bash)" .github/workflows/e2e.yaml || true
rg -n "raw\.githubusercontent\.com.*\|\s*(sh|bash)" .github/workflows/e2e.yaml || true
rg -n "run:\s*.*\|\s*(sh|bash)" .github/workflows/e2e.yaml || true
echo "=== Search for continue-on-error usage and where it is applied ==="
rg -n "continue-on-error" .github/workflows/e2e.yaml || true
echo "=== Inspect any socket wait loop / timeout handling ==="
rg -n "(socket|wait|nc -z|nc |/dev/tcp|localhost|timeout|sleep)" .github/workflows/e2e.yaml || trueRepository: txpipe/oura
Length of output: 7909
Pin CI GitHub Actions to immutable commit SHAs (and harden remote script execution).
- Unpinned
uses:tags:actions/checkout@v4,docker/setup-buildx-action@v3,docker/login-action@v3,docker/build-push-action@v6, andaws-actions/configure-aws-credentials@v4(both build and test jobs) should be pinned to commit SHAs. curl .../demeter-run/cli/.../install.sh | shexecutes a mutable remote script in CI—pin to a fixed commit/release and verify integrity (checksum/signature).continue-on-error: ${{ matrix.kind == 'n2c' }}at the job level suppresses alln2cregressions; scope it narrowly (e.g., step-level) or fail the job when the tunnel/socket can’t be established.- The
n2csocket wait loop (seq 1 30withsleep 2) doesn’t assert socket existence after the loop; if the socket never appears,timeout 1800 docker run ...can burn the full 30 minutes—add a post-loop failure when${RUNNER_TEMP}/sockets/node0.socketisn’t created.
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 17-17: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 17-17: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 20-20: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 23-23: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/e2e.yaml around lines 17 - 23, CI workflow uses floating
action tags and runs a remote install script insecurely; update every `uses:`
entry (e.g., actions/checkout@v4, docker/setup-buildx-action@v3,
docker/login-action@v3, docker/build-push-action@v6,
aws-actions/configure-aws-credentials@v4) to pinned commit SHAs, replace the
`curl .../demeter-run/cli/.../install.sh | sh` invocation with a download of a
specific release/commit and verify via checksum/signature before executing,
change `continue-on-error: ${{ matrix.kind == 'n2c' }}` from job-level to a
narrowly scoped step-level flag or remove it so failures bubble unless only the
specific tunnel/socket step is expected to be flaky, and after the `seq 1 30`
loop that waits for ${RUNNER_TEMP}/sockets/node0.socket add an explicit
existence check that fails the job immediately with a clear error if the socket
file was not created (to avoid burning full timeout in subsequent `timeout 1800
docker run ...`).
Source: Linters/SAST tools
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml:
- Around line 3-7: The peers array named "peers" currently lists a single host
("relay.cnode-m1.demeter.run:3000"), which is a single point of failure; update
the peers array to include at least one additional independent hostname/provider
(different domain) and port (e.g., another relay host) so the N2N leg has
redundancy; modify the peers value (the peers array) to contain multiple entries
on different hosts/providers and ensure entries use the same port format as
existing entries.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 00ffcb96-fdb1-44c0-acc5-1742bf553b50
📒 Files selected for processing (5)
.github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml.github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml.github/e2e/configs/aws-lambda-10-blocks.toml.github/e2e/configs/aws-s3-500-blocks.toml.github/e2e/configs/aws-sqs-10-blocks.toml
✅ Files skipped from review due to trivial changes (1)
- .github/e2e/configs/aws-s3-500-blocks.toml
🚧 Files skipped from review as they are similar to previous changes (3)
- .github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml
- .github/e2e/configs/aws-sqs-10-blocks.toml
- .github/e2e/configs/aws-lambda-10-blocks.toml
Comment on lines
+3
to
+7
| peers = [ | ||
| "relay.cnode-m1.demeter.run:3000", | ||
| "relay.cnode-m1.demeter.run:3001", | ||
| "relay.cnode-m1.demeter.run:3002", | ||
| ] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify whether N2N config redundancy is host-diverse across E2E scenarios.
# Expected: For resilient setups, each scenario should ideally contain >1 unique hostname.
set -euo pipefail
fd -p '.github/e2e/configs/*.toml' | while read -r f; do
if rg -n '^\[source\]' "$f" >/dev/null && rg -n 'type\s*=\s*"N2N"' "$f" >/dev/null; then
echo "=== $f ==="
rg -n 'relay|peers|type\s*=\s*"N2N"' "$f" -A6 -B1
python - "$f" <<'PY'
import re,sys
p=sys.argv[1]
txt=open(p).read()
hosts=re.findall(r'"([^":]+):\d+"', txt)
u=sorted(set(hosts))
print("unique_hosts:", u)
print("unique_host_count:", len(u))
PY
echo
fi
doneRepository: txpipe/oura
Length of output: 37
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== File contents: .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml ==="
nl -ba .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml | sed -n '1,120p'
echo
echo "=== Peer host extraction for N2N configs under .github/e2e/configs ==="
# List all TOML files, then filter to those with N2N source.type; for each, extract peer hostnames (host:port)
python3 - <<'PY'
import pathlib, re
root = pathlib.Path(".github/e2e/configs")
files = sorted(root.glob("*.toml"))
n2n = []
for p in files:
txt = p.read_text(encoding="utf-8")
if re.search(r'^\s*type\s*=\s*"N2N"\s*$', txt, re.M):
n2n.append(p)
def hosts_from_peers(txt):
# Capture hostname part of "hostname:port"
return re.findall(r'"([^":]+):\d+"', txt)
for p in n2n:
txt = p.read_text(encoding="utf-8")
hs = sorted(set(hosts_from_peers(txt)))
print(p.relative_to(pathlib.Path(".")))
print(" unique_hosts_count:", len(hs))
print(" unique_hosts:", hs)
PY
echo
echo "=== Exact peers list for the target file ==="
rg -n '^\s*peers\s*=\s*\[|relay\.cnode-m1\.demeter\.run|\"[^\"]+:\d+\"' .github/e2e/configs/assert-n2n-mainnet-tip-blocks.tomlRepository: txpipe/oura
Length of output: 194
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== File contents (numbered): .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml ==="
awk '{printf "%5d %s\n", NR, $0}' .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml | sed -n '1,140p'
echo
echo "=== Exact peers-related lines in target file ==="
rg -n '^\s*peers\s*=\s*\[|relay\.cnode-m1\.demeter\.run|\"[^\"]+:\d+\"' .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml || true
echo
echo "=== Unique peer hostnames for all N2N configs under .github/e2e/configs ==="
python3 - <<'PY'
import pathlib, re
root = pathlib.Path(".github/e2e/configs")
files = sorted(root.glob("*.toml"))
def is_n2n(txt: str) -> bool:
return re.search(r'^\s*type\s*=\s*"N2N"\s*$', txt, re.M) is not None
def hosts_from_peers(txt: str):
# hostname portion of "hostname:port"
return re.findall(r'"([^":]+):\d+"', txt)
for p in files:
txt = p.read_text(encoding="utf-8")
if not is_n2n(txt):
continue
hs = sorted(set(hosts_from_peers(txt)))
print(p.relative_to(pathlib.Path(".")))
print(" unique_hosts_count:", len(hs))
print(" unique_hosts:", hs)
PYRepository: txpipe/oura
Length of output: 1771
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import glob, re, pathlib
def parse_n2n_peers(path: str):
txt = pathlib.Path(path).read_text(encoding="utf-8")
if 'type = "N2N"' not in txt:
return None
pairs = re.findall(r'"([^":]+):(\d+)"', txt) # [(host, port), ...]
endpoints = [f"{h}:{p}" for h,p in pairs]
hosts = [h for h,_ in pairs]
return {
"file": path,
"peer_entries": len(endpoints),
"unique_endpoints": len(set(endpoints)),
"unique_hosts": len(set(hosts)),
"hosts": sorted(set(hosts)),
"endpoints": sorted(set(endpoints)),
}
paths = sorted(glob.glob(".github/e2e/configs/*.toml"))
for p in paths:
r = parse_n2n_peers(p)
if not r:
continue
print(f"\n=== {r['file']} ===")
print(f"peer_entries: {r['peer_entries']}")
print(f"unique_hosts: {r['unique_hosts']} ({r['hosts']})")
print(f"unique_endpoints: {r['unique_endpoints']} ({r['endpoints']})")
if r["peer_entries"] < 2 or r["unique_hosts"] < 2:
print("FLAG: insufficient redundancy/failover (add more peers, ideally on different hostnames/providers)")
PYRepository: txpipe/oura
Length of output: 2023
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import glob, re, pathlib
for p in sorted(glob.glob(".github/e2e/configs/*.toml")):
txt = pathlib.Path(p).read_text(encoding="utf-8")
if 'type = "N2N"' not in txt:
continue
pairs = re.findall(r'"([^":]+):(\d+)"', txt) # (host, port)
hosts = sorted(set(h for h,_ in pairs))
print(f"{p}: peer_entries={len(pairs)}, unique_hosts={len(hosts)}, hosts={hosts}")
PYRepository: txpipe/oura
Length of output: 903
Diversify N2N peers across independent hosts/providers (no failover today)
In .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml (lines 3-7), peers contains a single endpoint (relay.cnode-m1.demeter.run:3000). A host/domain incident can take down the entire N2N leg and make CI flaky. Add at least one additional peers entry on a different hostname/provider.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml around lines 3 - 7,
The peers array named "peers" currently lists a single host
("relay.cnode-m1.demeter.run:3000"), which is a single point of failure; update
the peers array to include at least one additional independent hostname/provider
(different domain) and port (e.g., another relay host) so the N2N leg has
redundancy; modify the peers value (the peers array) to contain multiple entries
on different hosts/providers and ensure entries use the same port format as
existing entries.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/e2e/configs/aws-s3-500-blocks.toml (1)
16-17:⚠️ Potential issue | 🟠 MajorAlign aws-s3-500-blocks.toml with SplitBlock+ParseCbor cloud-sink pipeline (or document why S3 differs).
AWS filter pipelines are inconsistent:
.github/e2e/configs/aws-lambda-10-blocks.toml: WorkStats → SplitBlock → ParseCbor.github/e2e/configs/aws-sqs-10-blocks.toml: WorkStats → SplitBlock → ParseCbor.github/e2e/configs/aws-s3-500-blocks.toml(lines 16-17): WorkStats → ParseCbor (no SplitBlock)If SplitBlock is required per the PR objectives, add
[[filters]] type = "SplitBlock"to the S3 config beforeParseCbor; otherwise add an explicit note explaining why S3 should omit it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/e2e/configs/aws-s3-500-blocks.toml around lines 16 - 17, The pipeline for the S3 config is missing the SplitBlock filter before ParseCbor; update the filters so the sequence is WorkStats → SplitBlock → ParseCbor by inserting a [[filters]] block with type = "SplitBlock" immediately before the existing ParseCbor filter, or if S3 must intentionally omit SplitBlock, add a short comment/documentation in that config explaining why SplitBlock is not applicable; reference the filter types "SplitBlock" and "ParseCbor" to locate the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/e2e/configs/aws-s3-500-blocks.toml:
- Around line 16-17: The pipeline for the S3 config is missing the SplitBlock
filter before ParseCbor; update the filters so the sequence is WorkStats →
SplitBlock → ParseCbor by inserting a [[filters]] block with type = "SplitBlock"
immediately before the existing ParseCbor filter, or if S3 must intentionally
omit SplitBlock, add a short comment/documentation in that config explaining why
SplitBlock is not applicable; reference the filter types "SplitBlock" and
"ParseCbor" to locate the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: de440e07-3eb3-4a0b-bdd9-26821f8b998f
📒 Files selected for processing (7)
.github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml.github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml.github/e2e/configs/assert-n2n-preview-origin-blocks.toml.github/e2e/configs/assert-n2n-preview-tip-blocks.toml.github/e2e/configs/aws-lambda-10-blocks.toml.github/e2e/configs/aws-s3-500-blocks.toml.github/e2e/configs/aws-sqs-10-blocks.toml
🚧 Files skipped from review as they are similar to previous changes (5)
- .github/e2e/configs/aws-sqs-10-blocks.toml
- .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml
- .github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml
- .github/e2e/configs/assert-n2n-preview-tip-blocks.toml
- .github/e2e/configs/assert-n2n-preview-origin-blocks.toml
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/e2e/scripts/setup-dmtr-socket.sh:
- Around line 25-28: The wait loop that polls for the socket at
"${RUNNER_TEMP}/sockets/node0.socket" can exit after the retries without the
socket actually existing; add a post-loop assertion that checks if the socket
file exists and if not prints an error to stderr and exits non-zero (e.g., with
exit 1) so the CI fails fast instead of letting downstream scripts like
run-test.sh hang; update the block around the seq loop to perform this existence
check and failure path immediately after the loop.
In `@src/bin/oura/run_daemon.rs`:
- Around line 99-101: When Arc::try_unwrap(daemon) returns Err the current code
silently skips calling daemon.teardown(), risking resource leaks; update the
block around Arc::try_unwrap(daemon) so that on Err(e) you log a warning
(including the Arc strong_count and/or debug info) indicating teardown could not
be performed and why, and keep the existing path that calls daemon.teardown()
when try_unwrap succeeds (referencing Arc::try_unwrap, the local variable
daemon, and daemon.teardown()). Ensure the warning uses the project's logger
(e.g., tracing::warn! or the existing logger) and includes enough context to
debug remaining references.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 12d3e203-9280-494c-83f4-9288062fb5f3
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (14)
.github/e2e/configs/u5c-mainnet-tip-blocks.toml.github/e2e/configs/u5c-preprod-tip-blocks.toml.github/e2e/configs/u5c-preview-tip-blocks.toml.github/e2e/scripts/run-test.sh.github/e2e/scripts/setup-dmtr-socket.sh.github/e2e/scripts/stop-dmtr-tunnel.sh.github/workflows/e2e.yamlCargo.tomlsrc/bin/oura/console.rssrc/bin/oura/dump.rssrc/bin/oura/main.rssrc/bin/oura/run_daemon.rssrc/bin/oura/watch.rssrc/daemon/mod.rs
✅ Files skipped from review due to trivial changes (2)
- .github/e2e/scripts/stop-dmtr-tunnel.sh
- .github/e2e/configs/u5c-preview-tip-blocks.toml
Comment on lines
+25
to
+28
| for _ in $(seq 1 30); do | ||
| if [ -S "${RUNNER_TEMP}/sockets/node0.socket" ]; then break; fi | ||
| sleep 2 | ||
| done |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Add a post-loop check to fail fast if the socket never appears.
The wait loop exits after ~60 seconds regardless of whether the socket was created. Without an assertion, run-test.sh will consume its full 30-minute timeout on a broken tunnel.
🛠️ Proposed fix
for _ in $(seq 1 30); do
if [ -S "${RUNNER_TEMP}/sockets/node0.socket" ]; then break; fi
sleep 2
done
+
+if [ ! -S "${RUNNER_TEMP}/sockets/node0.socket" ]; then
+ echo "ERROR: socket did not appear within 60 seconds" >&2
+ exit 1
+fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| for _ in $(seq 1 30); do | |
| if [ -S "${RUNNER_TEMP}/sockets/node0.socket" ]; then break; fi | |
| sleep 2 | |
| done | |
| for _ in $(seq 1 30); do | |
| if [ -S "${RUNNER_TEMP}/sockets/node0.socket" ]; then break; fi | |
| sleep 2 | |
| done | |
| if [ ! -S "${RUNNER_TEMP}/sockets/node0.socket" ]; then | |
| echo "ERROR: socket did not appear within 60 seconds" >&2 | |
| exit 1 | |
| fi |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/e2e/scripts/setup-dmtr-socket.sh around lines 25 - 28, The wait loop
that polls for the socket at "${RUNNER_TEMP}/sockets/node0.socket" can exit
after the retries without the socket actually existing; add a post-loop
assertion that checks if the socket file exists and if not prints an error to
stderr and exits non-zero (e.g., with exit 1) so the CI fails fast instead of
letting downstream scripts like run-test.sh hang; update the block around the
seq loop to perform this existence check and failure path immediately after the
loop.
scarmuega
force-pushed
the
ci/e2e-github-runners
branch
from
4b32d49 to
5a7f492
Compare
scarmuega
changed the title
scarmuega
force-pushed
the
ci/e2e-github-runners
branch
from
5a7f492 to
5aab106
Compare
scarmuega
force-pushed
the
fix/pipeline-finalize-exit
branch
from
7073f41 to
f540063
Compare
scarmuega
force-pushed
the
ci/e2e-github-runners
branch
from
5aab106 to
2401151
Compare
scarmuega
changed the base branch from
fix/pipeline-finalize-exit
to
refactor/run-daemon-stop-reason
Replaces the EKS-backed e2e workflow with one that runs entirely on github-hosted runners. Each leg runs the release image via `docker run` + `timeout` and gates on the process exit code; the new WorkStats filter gives each test a finalization policy so oura self-terminates after N blocks (exit 0), the Assert sink panics on a bad block (non-zero), and `timeout` guards a hang (124). - delete the `prepare` job (EKS kubeconfig / namespace / IRSA service account) - extract each config from its k8s ConfigMap into .github/e2e/configs/*.toml, rewritten for the oura v2 schema (peers/socket_path, top-level chain/intersect, LegacyV1 filter for the Assert sink, SplitBlock+ParseCbor for cloud sinks, RollbackBuffer for min_depth, WorkStats for finalization) - AWS creds via aws-actions/configure-aws-credentials (assuming the OuraE2ETest role) injected into the container instead of EKS IRSA - n2c uses a Demeter dmtrcli socket port-forward (placeholder; DMTR_API_KEY secret) and is continue-on-error until that path is finalized Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The three relay.cnode-m1.demeter.run endpoints map to different networks: :3000 = Mainnet, :3001 = PreProd, :3002 = Preview. Mainnet legs use :3000, preview n2n legs use :3002 (replacing the public IOG/world relays). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the run/dmtrcli logic out of e2e.yaml into .github/e2e/scripts/ (run-test.sh, setup-dmtr-socket.sh, stop-dmtr-tunnel.sh), invoked from the workflow steps. Scripts read the same env vars the steps already set; committed with the executable bit so they run after checkout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The U5C source connects over TLS via tonic/rustls. In builds that pull more than one rustls crypto provider (e.g. the `aws` feature brings in aws-lc-rs alongside ring), rustls has no process-default provider and panics on the first TLS handshake (`no process-level CryptoProvider available`). Install the ring provider once at startup, gated on the `u5c` feature. This only surfaced now because nothing exercised U5C in a multi-provider build; the manual u5c tests built `--features u5c` alone (single provider). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the manual u5c checks into the e2e suite as three `kind: u5c` legs against the Demeter utxorpc endpoints, using the standard exit-code gate via a WorkStats finalization policy (3 blocks -> exit 0); a #921-style regression (reset loop, no apply events) never finalizes -> timeout -> fail. API keys are parameterized as ${DMTR_UTXORPC_KEY_*} and resolved by envsubst from GitHub secrets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scarmuega
force-pushed
the
ci/e2e-github-runners
branch
from
2401151 to
7998fa5
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/e2e/scripts/run-test.sh:
- Around line 17-19: The script currently prints the fully resolved daemon.toml
(the echo/cat block that outputs "${RUNNER_TEMP}/daemon.toml"), which can leak
secrets; either remove that debug output entirely or replace it with a redaction
step that reads the resolved file, strips or masks known sensitive keys (e.g.,
dmtr-api-key and other credential fields) and then prints only the redacted
result or non-sensitive metadata (config path, test name, kind); update the
echo/cat block to call the redaction logic (or to be deleted) so CI logs never
contain raw secret values.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fbba5fd0-0d65-48cf-8fdf-8eb87636aaaa
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (29)
.github/e2e/assert-n2c-preview-origin-blocks.yaml.github/e2e/assert-n2c-preview-tip-blocks.yaml.github/e2e/assert-n2n-mainnet-origin-blocks.yaml.github/e2e/assert-n2n-mainnet-tip-blocks.yaml.github/e2e/assert-n2n-preview-origin-blocks.yaml.github/e2e/assert-n2n-preview-tip-blocks.yaml.github/e2e/aws-lambda-10-blocks.yaml.github/e2e/aws-s3-500-blocks.yaml.github/e2e/aws-sqs-10-blocks.yaml.github/e2e/configs/assert-n2c-preview-origin-blocks.toml.github/e2e/configs/assert-n2c-preview-tip-blocks.toml.github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml.github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml.github/e2e/configs/assert-n2n-preview-origin-blocks.toml.github/e2e/configs/assert-n2n-preview-tip-blocks.toml.github/e2e/configs/aws-lambda-10-blocks.toml.github/e2e/configs/aws-s3-500-blocks.toml.github/e2e/configs/aws-sqs-10-blocks.toml.github/e2e/configs/u5c-mainnet-tip-blocks.toml.github/e2e/configs/u5c-preprod-tip-blocks.toml.github/e2e/configs/u5c-preview-tip-blocks.toml.github/e2e/gcp-pubsub-10-blocks.yaml.github/e2e/scripts/run-test.sh.github/e2e/scripts/setup-dmtr-socket.sh.github/e2e/scripts/stop-dmtr-tunnel.sh.github/e2e/serviceaccount.yaml.github/workflows/e2e.yamlCargo.tomlsrc/bin/oura/main.rs
💤 Files with no reviewable changes (11)
- .github/e2e/aws-s3-500-blocks.yaml
- .github/e2e/gcp-pubsub-10-blocks.yaml
- .github/e2e/assert-n2n-preview-origin-blocks.yaml
- .github/e2e/aws-sqs-10-blocks.yaml
- .github/e2e/assert-n2n-preview-tip-blocks.yaml
- .github/e2e/aws-lambda-10-blocks.yaml
- .github/e2e/assert-n2c-preview-tip-blocks.yaml
- .github/e2e/assert-n2c-preview-origin-blocks.yaml
- .github/e2e/assert-n2n-mainnet-origin-blocks.yaml
- .github/e2e/assert-n2n-mainnet-tip-blocks.yaml
- .github/e2e/serviceaccount.yaml
✅ Files skipped from review due to trivial changes (5)
- .github/e2e/configs/u5c-preview-tip-blocks.toml
- .github/e2e/configs/assert-n2c-preview-origin-blocks.toml
- .github/e2e/configs/aws-s3-500-blocks.toml
- .github/e2e/configs/assert-n2n-preview-origin-blocks.toml
- .github/e2e/configs/assert-n2n-preview-tip-blocks.toml
🚧 Files skipped from review as they are similar to previous changes (10)
- .github/e2e/configs/assert-n2c-preview-tip-blocks.toml
- .github/e2e/configs/assert-n2n-mainnet-origin-blocks.toml
- .github/e2e/configs/u5c-preprod-tip-blocks.toml
- .github/e2e/scripts/stop-dmtr-tunnel.sh
- .github/e2e/configs/u5c-mainnet-tip-blocks.toml
- .github/e2e/configs/assert-n2n-mainnet-tip-blocks.toml
- src/bin/oura/main.rs
- .github/e2e/configs/aws-lambda-10-blocks.toml
- .github/e2e/configs/aws-sqs-10-blocks.toml
- .github/e2e/scripts/setup-dmtr-socket.sh
Comment on lines
+17
to
+19
| echo "----- resolved daemon.toml -----" | ||
| cat "${RUNNER_TEMP}/daemon.toml" | ||
| echo "--------------------------------" |
There was a problem hiding this comment.
Avoid printing resolved config that may contain embedded secrets.
After envsubst substitution, the daemon.toml may contain literal secret values (e.g., dmtr-api-key for u5c legs). Printing the full config to CI logs risks leaking credentials if GitHub's masking fails or if secrets appear in an unexpected format.
Consider redacting known sensitive keys before printing, or removing the debug output entirely in favor of logging only non-sensitive metadata (config path, test name, kind).
🔒 Suggested fix: redact sensitive fields or remove debug output
envsubst < ".github/e2e/configs/${TEST_NAME}.toml" > "${RUNNER_TEMP}/daemon.toml"
-echo "----- resolved daemon.toml -----"
-cat "${RUNNER_TEMP}/daemon.toml"
-echo "--------------------------------"
+echo "----- resolved daemon.toml (${TEST_NAME}) -----"
+# Redact sensitive keys before printing to avoid leaking secrets
+sed -E 's/(dmtr-api-key\s*=\s*)"[^"]*"/\1"***REDACTED***"/g' "${RUNNER_TEMP}/daemon.toml"
+echo "------------------------------------------------"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/e2e/scripts/run-test.sh around lines 17 - 19, The script currently
prints the fully resolved daemon.toml (the echo/cat block that outputs
"${RUNNER_TEMP}/daemon.toml"), which can leak secrets; either remove that debug
output entirely or replace it with a redaction step that reads the resolved
file, strips or masks known sensitive keys (e.g., dmtr-api-key and other
credential fields) and then prints only the redacted result or non-sensitive
metadata (config path, test name, kind); update the echo/cat block to call the
redaction logic (or to be deleted) so CI logs never contain raw secret values.
The n2c legs depended on a placeholder Demeter dmtrcli socket port-forward that isn't wired up yet. Remove them and their artifacts (the two configs, the setup/stop dmtrcli scripts, the socket mount and `continue-on-error` guard) so the suite is all-green and fails honestly on the remaining n2n/aws/u5c legs. Will reintroduce once the dmtrcli infra is in place. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OuraE2ETest role's trust path no longer matches static IAM user keys (the `security token invalid` failures). Switch the aws legs to keyless OIDC: add `id-token: write` and drop `aws-access-key-id`/`aws-secret-access-key` so configure-aws-credentials uses AssumeRoleWithWebIdentity. Requires (AWS side): an IAM OIDC provider for token.actions.githubusercontent.com and a trust policy on OuraE2ETest allowing this repo's workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The AwsS3 sink stores raw block CBOR and only accepts Record::CborBlock; the ParseCbor filter (copied from a stale example) turned it into a ParsedBlock, so the sink failed with 'config error'. S3 takes the CborBlock straight from the source — no parse filter. (SQS/Lambda keep SplitBlock + ParseCbor since those sinks consume per-tx ParsedTx records.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With OIDC the credential env is set by configure-aws-credentials; run-test.sh now forwards any AWS_* vars present rather than hardcoding AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/etc. Drops the now-unused KIND env. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The default-provider install was gated behind the u5c feature, but the multi-provider panic affects any TLS path that relies on rustls' process default (gcp/elasticsearch/hydra/mithril via reqwest/tungstenite), not just u5c. Make rustls a non-optional dep and run the install for every build so no feature combination can hard-crash on the first TLS handshake. The install is idempotent (the Result is ignored), so it's a no-op when a provider is already set, and the AWS SDK still uses its own provider. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These macOS junk files were swept in by an earlier `git add -A`. Untrack all five and add a .gitignore rule so they don't return. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The e2e suite ran on an external EKS cluster: build an image, assume an AWS role,
kubectl applya Job per test, gate onkubectl wait. This migrates it to run entirely on GitHub-hosted runners and deletes the cluster dependency. The configs are also rewritten for the oura v2 schema — they had rotted on v1 (peers/socket_path, top-level[chain]/[intersect], filter pipelines).Changes
preparejob (EKS kubeconfig/namespace/IRSA). Each leg runs the release image viadocker run+timeout; exit 0 = pass (WorkStats finalize, feat: add work_stats filter with finalization policy #928 + fix: bump gasket to 0.10 and derive exit code from finalization #929), non-zero = fail (Assert panic), 124 = hang..github/e2e/configs/*.toml, v2 schema,WorkStatsfinalization per leg; n2n legs point at the per-network Demeter relays (:3000mainnet /:3002preview)..github/e2e/scripts/instead of inlined YAML.tmp_u5c_testchecks, converted to the standard gate; doubles as a x=grpc error when using u5c demeter source #921 regression test. Keys parameterized as${DMTR_UTXORPC_KEY_*}secrets. Includes the rustls default-CryptoProvider fix the u5c TLS path needs in multi-provider builds.dmtrclisocket port-forward (placeholder commands;DMTR_API_KEYsecret) and arecontinue-on-erroruntil that path is finalized.Secrets required
DMTR_UTXORPC_KEY_{MAINNET,PREPROD,PREVIEW},DMTR_API_KEY, and working AWS creds for theOuraE2ETestrole (the last live run failed role-assumption: "security token invalid").Verification
workflow_dispatchrun validated the mechanics (build + image + streaming); the remaining reds were the AWS creds above, the Assert sink on Conway-era blocks (separate issue), and the placeholder n2c legs.🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
Chores
New Features