[runtime-failure-observer] Inline curl calls and require fetched evidence before opening PRs#1612
Merged
Merged
Conversation
The observer agent's shell commands were intermittently denied. The
copilot harness authorizes a command only by its first token, but the
prompt instructed the agent to pre-bind URLs (`url=...` then
`curl "$url"`) and to loop over definitions with `for`. Those forms
start with an assignment or keyword, so the harness rejected them with
"Permission denied and could not request permission from user" even
though the firewall allowlist contains .dev.azure.com and
.helix.dot.net. Across the three real runs this produced three
different outcomes (worked around, noop, and a false report_incomplete
that blamed the firewall).
Changes to runtime-failure-observer.agent.md (prompt body, imported at
runtime via {{#runtime-import}}, so no lock recompile needed):
- Rule 11 now requires every shell command to begin with an
allow-listed program; inline URLs into `curl ... -o file`, no
variable pre-bind, no loops. Step 1, Step 2, and the Step 4 dedup
cache snippet are rewritten to match.
- New Step 0 preflight proves egress with one inlined curl and, on
failure, emits an accurate report_incomplete (harness command
authorization, not firewall) instead of misdiagnosing the firewall.
- New rule 6b forbids opening a PR unless the build timeline and Helix
console were actually downloaded this run; no citing build ids,
Helix GUIDs, exit codes, or stderr from memory or inference.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kotlarmilos
changed the title
There was a problem hiding this comment.
Pull request overview
Updates the runtime-failure-observer agent instructions to avoid non-deterministic “permission denied” failures caused by the harness authorizing shell commands solely by their first token, and to require citing only evidence fetched during the current run before opening PRs.
Changes:
- Tightens the “Never assume” rule to prohibit reconstructing build IDs/URLs/exit codes/stderr when fetches fail.
- Replaces variable pre-binding /
forloops with “inline URL”curl ... -o filecommand forms. - Adds a preflight
curlcheck to validate egress/authorization before scanning pipelines.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/runtime-failure-observer.agent.md | Refines agent operational rules and command examples to align with first-token authorization and “fetched evidence only” requirements. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 4
vitek-karas
approved these changes
Jun 11, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
matouskozak
reviewed
Jun 14, 2026
Member
Author
|
/ba-g #1623 |
Member
Author
|
re-approval needed |
vitek-karas
approved these changes
Jun 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
runtime-failure-observeragent ran non-deterministically because the copilot harness authorizes a shell command only by its first token, yet the prompt told the agent to pre-bind URLs and loop withfor. Those forms were denied withPermission deniedeven though the firewall already allows.dev.azure.com/.helix.dot.net.This PR rewrites rule 11 to require every command to start with an allow-listed program and inline each URL into a single
curl ... -o file.