Skip to content

Commit 4124d9f

Browse files
authored
Add benchmark section for A2A FAIR Signposting
Added a section about the A2A FAIR Signposting benchmark runner, detailing its purpose, functionality, and scope.
1 parent e5e8c60 commit 4124d9f

1 file changed

Lines changed: 12 additions & 0 deletions

File tree

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,18 @@ Together, they form a clean separation of concerns:
184184

185185
---
186186

187+
## Benchmark: A2A FAIR Signposting (HTTP) — Compass validation runner
188+
189+
To continuously assess Compass’ validator behavior on real-world FAIR Signposting examples, we maintain an **offline, apple-to-apple benchmark runner** based on the **A2A FAIR Signposting HTTP scenarios**.
190+
191+
The runner replays harvested HTTP header fixtures (primarily RFC 8288 `Link` headers), converts them into the **WebLink model**, and validates them with Compass against scenario-specific expectations (relations and expected issues). This makes it easy to reproduce behavior across versions and detect regressions.
192+
193+
**Scope note:** The benchmark currently covers **HTTP scenarios only**. Inline HTML signposting is intentionally not included yet (that would mainly benchmark HTML parsing, not Compass’ WebLink validation). Network-based behavior (dereferencing, conneg, remote Link Set fetching) is also out of scope.
194+
195+
Benchmark runner repository: **https://github.com/qbicsoftware/compass-a2a-runner**
196+
197+
---
198+
187199
## Contributing
188200

189201
Compass is an open-source research software project and welcomes contributions

0 commit comments

Comments
 (0)