test suite access for implementers not using VC-API or DI EdDSA

[brentzundel]

Mesur.io is implementing Verifiable Credentials Data Model 2.0 and would like to include our implementation of the spec in the test suite, however we have no plans to implement VC-API or DI EdDSA.

It seems strange to us that the test suite has been built in such as way as to require those technologies in order to demonstrate interop with the data model.

What is the mechanism for demonstrating data-model interop for those organizations that have no need for VC-API and DI EdDSA?

[thereisnogabe]

I would like to recommend either (or both) changes:

1. The test suite is amended to simply test the data model (nothing around issuance, verification, or reliance on a web server or API). This could be similar to the did test suite -- something like simple JSON Schema based tests.
2. The test suite is modified to be an aggregate of tests from conforming securing specifications (Data Integrity and VC JOSE COSE), since demonstrating conformance with a given securing mechanism includes demonstrating conformance with the data model.

[BigBlueHat]

I think @decentralgabe's approach #2 could be done pretty quickly. We've mainly just lacked test suite collaborators to help us find what's missing for enveloped proofs. I'd be happy to setup more office hours calls, so we can tackle this together.

The JSON Schema approach could also be taken, but we'd loose a good deal in terms of ongoing testing and the ability to run interop tests.

The VC API bits are really just a handful of HTTP API endpoints that need to accept POST requests, then do whatever they're going to do (envelope, issue, validate, etc.) and respond. We should be able to use EnvelopedVerifiableCredential and EnvelopedVerifiablePresentation as wrappers when/where needed for moving those things over the wire.

Greg's also built a stub server for anyone using node: https://github.com/Wind4Greg/Server-for-VCs It supports far more things now, but there's a minimal subset in there we could provide as a boilerplate. That boilerplate could then be used to wrap other activities as needed.

Happy to setup a call to go through what's needed! And certainly, I'd be happy to start an office hours call back up through the end of the year, so we have a space to collaborate long(er) term. Sound like a place to start at least?

[thereisnogabe]

Discussed a bit earlier and I suggested that I work with you @BigBlueHat to integrate the VC JOSE COSE test suite with this suite to make approach 2 possible.

[PatStLouis]

@brentzundel We've added a tag for EnvelopingProofs, which can be passed with an implementation that doesn't use data integrity. You no longer need to support DI EdDSA to be conformant with this test suite. I will ensure this is properly documented.

The biggest concern seems to be leveraging the vc-api. I was not part of the group discussion when it was decided to be used for test-suites, however it seems there is now some implementers who want to move back on that decision. I've built an implementation that conforms with all 3 specifications (vcdm 2.0, vc-jose, di-eddsa) and would like to ensure that this new direction does not require me to change said implementation. (I assume same would go for all other implementers)

I also don't think the vcdm test suite should be aware of Data Integrity or VC-JOSE, but instead focus on embeded proof / enveloping proofs as defined by the specification. The issue with enveloping proof is that there is further processing required for the test client to access the signed payload (the vcdm part). For this test-suite to be applicable, the test client will need to first extract the payload, then run it through the data model testing.

The test suite currently does 3 things:

1. Ensure the issuing software doesn't accept non conformant input. <- agnostic to securing mechanism
2. Ensure the issuing software generates a conformant output (vc) <- depends on securing mechanism
3. Ensure the verifying software doesn't accept non conformant input OR detects non conformant input. <- depends on securing mechanism

If we can agree that for enveloping proofs, implementers MUST use vc-jose, this can be made slightly easier. For embeded proof there is no such requirements since the vcdm is directly exposed to the test client and this test suite will not make assertions on the securing mechanism.

I'm happy to help find a way to make this work, perhaps by providing an additional mechanism to run the tests that outputs to the same report format. @decentralgabe I saw some docker based testing floating around you were working on which I liked. This is a direction I would be interested in exploring. The traceability group was also discussing implementers to provide static fixtures the last time I checked. This isn't ideal since it doesn't really test the implementation, only a static document.

[PatStLouis]

I will also add that so far using the vc-api is the only consistent testing methodology I've seen used for VC's. The concept of having a common api testing interface is also a design I've seen used successfully in other testing frameworks, such as the Agent Test Harness at OWF.