Skip to content

methodology.md

Latest commit

 

History

History
77 lines (54 loc) · 2.54 KB

methodology.md

File metadata and controls

77 lines (54 loc) · 2.54 KB

Methodology

Goal

The goal of this repository is to determine whether coding agents exhibit a converging runtime behavior pattern that can be formalized independently of any single product, model vendor, or orchestration framework.

What Counts As A Runtime Pattern

A runtime pattern is a repeated behavior or control structure that shapes how an agent executes work. The pattern must affect execution semantics rather than only presentation or branding.

Examples:

  • how tool calls are initiated and reintegrated,
  • how completion is attempted and validated,
  • how failures re-enter the loop,
  • how state transitions are represented,
  • how context compaction is triggered,
  • how subagents are bounded and observed.

Counterexamples:

  • visual styling,
  • marketing taxonomy,
  • vendor-specific naming,
  • domain-specific repository rules.

Source Hierarchy

Preferred sources, in order:

  1. source code,
  2. official documentation,
  3. design discussions and issue threads,
  4. demos and observed behavior,
  5. secondary commentary.

Whenever evidence comes from observation rather than code or docs, label it clearly.

Evidence Collection Rules

For each system, collect evidence against the same analytical axes:

  1. execution loop,
  2. completion semantics,
  3. tool failure model,
  4. state machine,
  5. context management,
  6. subagents and delegation,
  7. policy and hooks,
  8. UI and telemetry coupling,
  9. permission model,
  10. extensibility surface.

Separating Generic From Product-Specific

Use these tests:

  1. Does the behavior solve a generic runtime problem?
  2. Does it appear in more than one system or seem structurally reusable?
  3. Can it be expressed without referencing one product shell?
  4. Would removing the behavior materially weaken robustness?

If the answer is no, the behavior likely belongs in product-specific analysis rather than in the shared runtime contract.

Confidence Labels

  • high-confidence convergence: repeated across multiple systems with strong evidence.
  • medium-confidence convergence: visible in several systems but with partial evidence or inconsistent implementation.
  • emerging but unstable: promising pattern with limited or shifting evidence.
  • speculative proposal: design hypothesis not yet strongly supported by observed implementations.

Revision Rules

  1. Prefer narrowing a claim over defending an overstated one.
  2. Update pattern pages when a new system weakens or complicates an earlier conclusion.
  3. Preserve open questions rather than forcing false certainty.
  4. Keep the synthesis honest about where convergence stops.
⚡