Merge pull request #1576 from finos/fdc3-new-conformance

Conformance 2.2: Port to Monorepo

Author: kriswest

CHANGELOG.md

@@ -14,6 +14,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 * Added details of and procedures for resolving fully-qualified appIds and unqualified appIds in the API and Bridging Parts of the Standard. ([#1523](https://github.com/finos/FDC3/pull/1523))
 
+- Ported FDC3 Conformance Project as-is into the FDC3 Monorepo, just including minimal fixes for typescript compilation. ([#1576](https://github.com/finos/FDC3/pull/1576))
+
 ### Changed
 
 ### Deprecated

package-lock.json

@@ -29,7 +29,8 @@
     "packages/fdc3-commonjs",
     "toolbox/fdc3-for-web/reference-ui",
     "toolbox/fdc3-for-web/demo",
-    "toolbox/fdc3-workbench"
+    "toolbox/fdc3-workbench",
+    "toolbox/fdc3-conformnance"
   ],
   "scripts": {
     "clean": "npm run clean --workspaces --if-present",
@@ -39,7 +40,7 @@
     "report": "nyc report --reporter json-summary --report-dir nyc-coverage-report --exclude-after-remap false --temp-dir coverage",
     "lint": "npm run lint --workspaces --if-present",
     "syncpack": "npm exec syncpack fix-mismatches --types 'local'",
-    "dev": "concurrently \"cd toolbox/fdc3-workbench && npm run dev\" \"cd toolbox/fdc3-for-web/reference-ui && npm run dev\" \"cd toolbox/fdc3-for-web/demo && npm run dev\"",
+    "dev": "concurrently \"cd toolbox/fdc3-conformance && npm run dev\" \"cd toolbox/fdc3-workbench && npm run dev\" \"cd toolbox/fdc3-for-web/reference-ui && npm run dev\" \"cd toolbox/fdc3-for-web/demo && npm run dev\"",
     "prepare": "husky"
   },
   "prettier": {
@@ -65,4 +66,4 @@
   "lint-staged": {
     "**/*": "prettier --write --ignore-unknown"
   }
-}
+}

package.json

@@ -0,0 +1,104 @@
+<img align="right"  title="FDC3 Conformance Framework" width="200px" src="https://landscape.finos.org/logos/fdc3-conformance-framework.svg">
+
+# FDC3 Conformance Framework
+
+A framework for testing whether desktop containers implement the [FDC3 standard](https://fdc3.finos.org/).
+
+The tests implemented for each FDC3 version are based on the FDC3 Conformance tests specifications defined by the FDC3 maintainers and hosted on the [FDC3 website](../../website/docs/api/conformance/Overview.md). 
+
+Tests in this module currently implement versions 2.0 and 2.2 of the FDC3 Standard.  For old 1.2 conformance testing, please see the [archived FDC3 Conformance project](https://github.com/finos/FDC3-Conformance-Framework) 
+
+## What Is It?
+
+- There are many vendors implementing Desktop Agents for the FDC3 standard.
+- Developers building apps interoperating with the FDC3 standard want to be sure that their apps will work with every Desktop Agent.
+- This suite of conformance tests allows for programmatic verification of an FDC3 Desktop Agent implementation.
+
+## Why Is This Important?
+
+✔️ **Multiple Platforms**
+
+It can be annoying for users to find that features in their apps that they rely on don't work when changing between Desktop Agents.  Hopefully these conformance tests will add security around that.
+
+✔️ **Marketing**
+
+FINOS has created _badges_ to indicate conformance with the FDC3 standard.   By passing the conformance tests and joining the conformance program, firms are able to use those badges in their own marketing materials.
+
+✔️ **Backwards Compatibility**
+
+There are multiple versions of the FDC3 standard.  A Desktop Agent could host apps written in _any one of these versions_.  For that reason, it's important that Desktop Agents are tested to make sure that they keep on supporting apps written against older versions of FDC3.
+
+## How Does It Work?
+
+There are two main parts to conformance:
+  
+  - Running the tests _locally_
+  - Joining the Conformance Program
+
+### Running The Conformance Tests
+
+You can either run the hosted conformance tests listed in the FINOS App Directory, or run them on your local machine (useful if you are making changes).  Instructions for doing either of those things are here:
+
+#### Running From fdc3.finos.org
+
+You can run the FDC3 conformance tests straight from the FDC3 website.  If you have a Desktop Agent supporting the [AppD v2 standard](https://fdc3.finos.org/docs/app-directory/spec), you can run the tests from the following URL:
+
+ - [https://fdc3.finos.org/toolbox/fdc3-conformance/directory.v2.json](https://fdc3.finos.org/toolbox/fdc3-conformance/directory.v2.jso)
+
+#### Local Installation
+
+Alternatively, you can run the conformance suite from your local machine by following these steps:
+
+1.  **Check Out This Repo**
+
+The FDC3 project is a monorepo and the `toolbox/fdc3-conformance` folder contains the conformance tests.  
+
+2. **Install Dependencies**
+
+In order to get started, install all the dependencies with:
+
+```sh
+npm install
+```
+
+3.  **Run The Apps Locally**
+
+To run the conformance suite (and all of the other FDC3 project components such as the workbench and demo desktop agent) locally on port 3001:
+
+```sh
+npm run dev
+```
+
+4.  **Set Up Your Desktop Agent**
+
+You will need to set up your desktop agent so that it has an _App Directory containing all the conformance apps_.   
+This step is vendor-dependent, but examples of `AppD` records can be found in the `directory` folder.
+
+### Running The Tests
+
+Once you have configured your desktop agent correctly, open the conformance app and select the tests you wish to run:
+
+<img src="static/selecting.png" alt="Selecting Tests" width="400px" />
+
+The tests run and produce an output in the window.  Failure are shown with a stack trace in red:
+
+<img src="static/failing.png" alt="Running Tests" width="400px" />
+
+Successful runs look something like this:
+
+<img src="static/running.png" alt="Success" width="400px" />
+
+### Joining The Conformance Program
+
+If you've had a clean run of all the tests locally, why not join the conformance program?
+
+[Instructions to join the Certified FDC3 Desktop Agent Conformance Program](instructions.md)
+
+Once you have followed these steps, you will be allowed to display the FDC3 Compliance Badges within your marketing literature.
+
+<img src="terms-conditions/graphics/1.2/2022_FDC3_CompliantBadge_1.2.png" alt="Conformance with FDC3 1.2" width="100px"><img src="terms-conditions/graphics/2.0/2022_FDC3_CompliantBadge_2.0.png" alt="Conformance with FDC3 2.0"  width="100px">
+
+### Which Desktop Agents Are Conformant?
+
+We publish the details of conformant desktop agents on the [FDC3 Home Page](https://fdc3.finos.org#conformance).  Please check there to find out who FINOS has certified! 
+

toolbox/fdc3-conformance/README.md

@@ -0,0 +1,13 @@
+import globals from 'globals';
+import pluginJs from '@eslint/js';
+import tseslint from 'typescript-eslint';
+import eslintConfigPrettier from 'eslint-config-prettier';
+
+/** @type {import('eslint').Linter.Config[]} */
+export default [
+  { files: ['**/*.{js,mjs,cjs,ts}'] },
+  { languageOptions: { globals: globals.browser } },
+  pluginJs.configs.recommended,
+  ...tseslint.configs.recommended,
+  eslintConfigPrettier,
+];

toolbox/fdc3-conformance/eslint.config.mjs

@@ -0,0 +1,52 @@
+# How to Apply For FDC3 Certification for a Desktop Agent
+
+## 1. Run The Conformance Tests Yourself.
+
+See [Readme](README.md) about how to do this.  
+ 
+## 2. Raise An Issue On This project
+
+[Raise An Issue](https://github.com/finos/FDC3-Conformance-Framework/issues) on the FDC3 Conformance Project.  
+
+In all likelihood, you will need to liase with the maintainers of this project in order to get the conformance tests to work with your desktop agent.  
+
+This will also kick off the process of applying for the "FDC3 Certified" mark.
+
+- Provide details in the issue about how you configured and ran the conformance tests in step 1 above.
+- Provide the test results (number of passes / fails, details of failed tests, versions used) in the issue.
+
+## 3.  Replication
+
+Maintainers of the FDC3 Conformance Framework will aim to replicate your test results.  You must work with them to provide the correct version(s) of your software and resolve any issues around licensing that may arise in order  that they can proceed.  
+
+## 4.  Fees
+
+Fees for certification are:
+
+|FINOS Membership Level        |Fee per Unit*     |
+|------------------------------|------------------|
+|Platinum|Free (unlimited units) |
+|Gold    |Free (up to 3 units each membership year) |
+|Silver |Free (first unit each membership year)|
+|Additional Units / Non Members |$20,000 per unit|
+
+These fees are to cover work done by the FINOS for the formal certification process and badge issuance. 
+
+*A _unit_ here refers to a particular vendor software version and FDC3 version _combination_, as described in the [Terms and Conditions](terms-conditions/FDC3-Certified-Terms.md)
+
+## 5.  Failed Tests
+
+If your desktop agent fails any of the conformance tests, the maintainers will inform you via the issue you opened in order that you can make fixes to your software and retry the process.
+
+## 6.  Terms
+
+Once your desktop agent has passed the complete suite of tests, you will be required to accept the [Terms and Conditions](terms-conditions/FDC3-Certified-Terms.md) via the completion of the [Participation Form](terms-conditions/participation-form.md).
+
+## 7.  Badge
+
+On completion of the above steps you will be given your certificate via the [Credly](https://cred.ly) platform for use in your own materials / publicity.
+
+## 8.  Re-Certification
+
+Certification only applies to a single version of your software product. FINOS reserves the right to allow certification to carry over at no additional cost for minor/maintenance version changes of your desktop agent if no backward incompatible changes are introduced in those versions. 
+