Merge pull request #27 from softwarepub/create-validation-report

Create basic human-readable validation reports.

Author: zyzzyxdonta

README.md

@@ -6,48 +6,20 @@ SPDX-FileContributor: David Pape
 
 # Software CaRD Policies
 
-This repository contains example policies developed as part of the Software CaRD project, as well as a validator tool.
-
-## Conventions
-
-All examples in this repository use the following namespace prefix bindings:
-
-```turtle
-@prefix codemeta: <https://doi.org/10.5063/schema/codemeta-2.0#> .
-@prefix owl: <http://www.w3.org/2002/07/owl#> .
-@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
-@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
-@prefix sc: <https://schema.software-metadata.pub/software-card/2025-01-01/#> .
-@prefix scex: <https://schema.software-metadata.pub/software-card/2025-01-01/examples/#> .
-@prefix scimpl: <https://schema.software-metadata.pub/software-card/2025-01-01/implementation/#> .
-@prefix schema: <https://schema.org/> .
-@prefix sh: <http://www.w3.org/ns/shacl#> .
-@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
-```
-
-For Software CaRD, the prefixes
-[`sc:`](https://schema.software-metadata.pub/software-card/2025-01-01/#),
-[`scex:`](https://schema.software-metadata.pub/software-card/2025-01-01/examples/#), and
-[`scimpl:`](https://schema.software-metadata.pub/software-card/2025-01-01/implementation/#)
-were established and are used for the following purposes:
-
-- `sc:` contains terms exposed to users
-- `scex:` contains example uses of `sc:` and `sh:` terms
-- `scimpl:` contains internal implementation details
-
-The associated IRIs currently don't exist.
-A [search on prefix.cc](https://prefix.cc/sc) reveals prior usage of the prefix `sc:` by projects which seem to be
-defunct.
+This repository contains the `software_card_policies` Python library as well as the associated command line program
+`sc-validate` and example policies.
+The software was written as part of the [Software CaRD](https://helmholtz-metadaten.de/en/inf-projects/softwarecard)
+project.
 
 ## `sc-validate`
 
-A program that validates a given metadata file using a set of configurable policies.
+A command line program that validates a given metadata file using a set of configurable policies.
 
-The selection of policies to use can be configured via [`config.toml`](config.toml).
+The selection of policies can be configured via [`config.toml`](config.toml).
 Policies can be loaded using any of the protocols supported by
 [RDFlib's `Graph.parse` method](https://rdflib.readthedocs.io/en/stable/apidocs/rdflib.html#rdflib.graph.Graph.parse)
 (e.g. local files, http, ...).
-All of the given policies are loaded and merged into one RDF graph (union of all triples of the parts).
+All of the given policies are loaded and unioned into one RDF graph.
 
 Policies can be implemented in a configurable fashion by defining an `sc:Parameter` and using it in place of a literal
 or list.
@@ -66,7 +38,7 @@ python -m pip install -e .
 
 ### Run
 
-Start a webserver hosting the policy files (run it in the background or use a separate terminal window):
+Start a webserver hosting the example policy files (run it in the background or use a separate terminal window):
 
 ```bash
 python -m http.server -b 127.0.0.1 -d examples/
@@ -79,13 +51,45 @@ sc-validate examples/data/hermes.ttl
 ```
 
 This will validate [`hermes.ttl`](examples/data/hermes.ttl) using the policies defined in [`config.toml`](config.toml)
-and print the result to the screen.
-If run in debug mode (with `--debug`), the following files are written to the current working directory:
+and print a validation report to the screen.
+If run in debug mode (with `--debug`), the report is more verbose, and the following files are written to the current
+working directory:
 
 - `debug-input-data.ttl`: the input data
 - `debug-shapes-processed.ttl`: the parameterized and combined policies
 - `debug-validation-report.ttl`: the detailed SHACL validation report (`sh:ValidationReport`)
 
+## Conventions
+
+All examples in this repository use the following namespace prefix bindings:
+
+```turtle
+@prefix codemeta: <https://doi.org/10.5063/schema/codemeta-2.0#> .
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix sc: <https://schema.software-metadata.pub/software-card/2025-01-01/#> .
+@prefix scex: <https://schema.software-metadata.pub/software-card/2025-01-01/examples/#> .
+@prefix scimpl: <https://schema.software-metadata.pub/software-card/2025-01-01/implementation/#> .
+@prefix schema: <https://schema.org/> .
+@prefix sh: <http://www.w3.org/ns/shacl#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+```
+
+For Software CaRD, the prefixes
+[`sc:`](https://schema.software-metadata.pub/software-card/2025-01-01/#),
+[`scex:`](https://schema.software-metadata.pub/software-card/2025-01-01/examples/#), and
+[`scimpl:`](https://schema.software-metadata.pub/software-card/2025-01-01/implementation/#)
+were established and are used for the following purposes:
+
+- `sc:` contains terms exposed to users
+- `scex:` contains example uses of `sc:` and `sh:` terms
+- `scimpl:` contains internal implementation details
+
+The associated IRIs currently don't exist.
+A [search on prefix.cc](https://prefix.cc/sc) reveals prior usage of the prefix `sc:` by projects which seem to be
+defunct.
+
 ## Acknowledgments
 
 [Software CaRD](https://helmholtz-metadaten.de/en/inf-projects/softwarecard) (`ZT-I-PF-3-080`) is funded by the

pyproject.toml

@@ -14,8 +14,9 @@ dependencies = [
   "rdflib>=7.1.1",
   "pydantic>=2.9.2",
   "pydantic-settings[toml]>=2.6.1",
+  "jinja2>=3.1.6",
 ]
-requires-python = ">=3.10"
+requires-python = ">=3.10,<3.13"  # problems compiling pyduktape2 when using Python 3.13
 authors = [
   {name = "David Pape", email = "d.pape@hzdr.de"},
 ]
@@ -30,11 +31,11 @@ classifiers = [
 [project.scripts]
 sc-validate = "sc_validate.__main__:main"
 
-[tool.ruff]
-target-version = "py310"
-
 [tool.ruff.lint]
 select = ["E", "F", "I", "N", "W"]
 
+[tool.ruff.lint.per-file-ignores]
+"src/sc_validate/namespaces.py" = ["N815"]  # mixed case class parameters
+
 [tool.setuptools_scm]
 version_file = "src/sc_validate/_version.py"

src/sc_validate/__main__.py

@@ -17,6 +17,7 @@
     read_rdf_resource,
     validate_graph,
 )
+from sc_validate.report import create_report
 
 
 def path_or_url(path: str) -> pathlib.Path | str:
@@ -74,8 +75,10 @@ def main():
     if arguments.debug:
         validation_graph.serialize("debug-validation-report.ttl", "turtle")
 
+    report = create_report(validation_graph, debug=arguments.debug)
+    print(report)
+
     if not conforms:
-        print("validation failed", file=sys.stderr)
         sys.exit(1)
 
 

src/sc_validate/namespaces.py

@@ -0,0 +1,53 @@
+# SPDX-FileCopyrightText: 2025 Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileContributor: David Pape
+
+from rdflib.namespace import OWL, RDF, RDFS, SDO, SH, XSD, DefinedNamespace, Namespace
+from rdflib.term import URIRef
+
+
+class SC(DefinedNamespace):
+    """The Software CaRD schema."""
+
+    _NS = Namespace("https://schema.software-metadata.pub/software-card/2025-01-01/#")
+
+    Parameter: URIRef
+
+    parameterType: URIRef
+    parameterConfigPath: URIRef
+    parameterDefaultValue: URIRef
+
+
+class SCIMPL(DefinedNamespace):
+    """Software CaRD implementation details."""
+
+    _NS = Namespace(
+        "https://schema.software-metadata.pub/software-card/2025-01-01/implementation/#"
+    )
+
+
+class SCEX(DefinedNamespace):
+    """Software CaRD example components."""
+
+    _NS = Namespace(
+        "https://schema.software-metadata.pub/software-card/2025-01-01/examples/#"
+    )
+
+
+CODEMETA = Namespace("https://doi.org/10.5063/schema/codemeta-2.0#")
+"""The Codemeta schema. See: https://codemeta.github.io/"""
+
+
+PREFIXES = {
+    "codemeta": CODEMETA,
+    "owl": OWL,
+    "rdf": RDF,
+    "rdfs": RDFS,
+    "sc": SH,
+    "scex": SCEX,
+    "scimpl": SCIMPL,
+    "schema": SDO,
+    "sh": SH,
+    "xsd": XSD,
+}
+"""Default namespace prefix bindings."""

src/sc_validate/rdf.py

@@ -8,29 +8,15 @@
 from pyshacl import validate
 from rdflib import BNode, Graph, Literal
 from rdflib.collection import Collection
-from rdflib.namespace import RDF, Namespace
+from rdflib.namespace import RDF
 
-# Just for better readability of serialized linked data.
-BINDINGS = {
-    "codemeta": "https://doi.org/10.5063/schema/codemeta-2.0#",
-    "owl": "http://www.w3.org/2002/07/owl#",
-    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
-    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
-    "sc": "https://schema.software-metadata.pub/software-card/2025-01-01/#",
-    "scex": "https://schema.software-metadata.pub/software-card/2025-01-01/examples/#",
-    "scimpl": "https://schema.software-metadata.pub/software-card/2025-01-01/implementation/#",
-    "schema": "https://schema.org/",
-    "sh": "http://www.w3.org/ns/shacl#",
-    "xsd": "http://www.w3.org/2001/XMLSchema#",
-}
-
-SC = Namespace("https://schema.software-metadata.pub/software-card/2025-01-01/#")
+from sc_validate.namespaces import PREFIXES, SC
 
 
 def read_rdf_resource(source: pathlib.Path | str) -> Graph:
     graph = Graph()
     graph.parse(source)
-    for prefix, iri in BINDINGS.items():
+    for prefix, iri in PREFIXES.items():
         graph.bind(prefix, iri)
     return graph
 

src/sc_validate/report.py

@@ -0,0 +1,88 @@
+# SPDX-FileCopyrightText: 2025 Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileContributor: David Pape
+
+from dataclasses import dataclass
+from enum import Enum
+from typing import List
+
+from jinja2 import Environment, PackageLoader, select_autoescape
+from rdflib import Graph, Literal
+from rdflib.namespace import RDF, SH
+from rdflib.term import URIRef
+
+
+# TODO: This only works for constraints of type NodeShape. Is this enough?
+@dataclass
+class Policy:
+    name: str
+    description: str
+
+    @classmethod
+    def from_graph(cls, reference: URIRef, graph: Graph):
+        name = graph.value(reference, SH.name, None)
+        description = graph.value(reference, SH.description, None)
+        return cls(name=name, description=description)
+
+
+class Severity(Enum):
+    INFO = 1
+    WARNING = 2
+    VIOLATION = 3
+    OTHER = 4
+
+    def __str__(self):
+        return self.name.title()
+
+    @classmethod
+    def from_graph(cls, reference: URIRef, graph: Graph):
+        if reference == SH.Info:
+            return cls.INFO
+        if reference == SH.Warning:
+            return cls.WARNING
+        if reference == SH.Violation:
+            return cls.VIOLATION
+        return cls.OTHER
+
+
+@dataclass
+class ValidationResult:
+    severity: Severity
+    message: str
+    source_policy: Policy
+
+    @classmethod
+    def from_graph(cls, reference: URIRef, graph: Graph):
+        severity = graph.value(reference, SH.resultSeverity, None)
+        message = graph.value(reference, SH.resultMessage, None)
+        source_policy = graph.value(reference, SH.sourceShape, None)
+        return cls(
+            severity=Severity.from_graph(severity, graph),
+            message=message,
+            source_policy=Policy.from_graph(source_policy, graph),
+        )
+
+
+@dataclass
+class ValidationReport:
+    conforms: bool
+    results: List[ValidationResult]
+
+    @classmethod
+    def from_graph(cls, reference: URIRef, graph: Graph):
+        conforms = (reference, SH.conforms, Literal(True)) in graph
+        results = graph.objects(reference, SH.result)
+        return cls(
+            conforms=conforms,
+            results=[ValidationResult.from_graph(result, graph) for result in results],
+        )
+
+
+def create_report(validation_graph: Graph, debug=False) -> str:
+    shacl_report, *_ = validation_graph.subjects(RDF.type, SH.ValidationReport)
+    validation_report = ValidationReport.from_graph(shacl_report, validation_graph)
+    environment = Environment(
+        loader=PackageLoader("sc_validate"), autoescape=select_autoescape()
+    )
+    template = environment.get_template("report.j2")
+    return template.render(validation_report=validation_report, debug=debug)

src/sc_validate/templates/report.j2

@@ -0,0 +1,17 @@
+{#
+SPDX-FileCopyrightText: 2025 Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
+SPDX-License-Identifier: Apache-2.0
+SPDX-FileContributor: David Pape
+#}
+{%- if validation_report.conforms -%}
+Validation succeeded!
+{%- else -%}
+Validation failed!
+
+{% for validation_result in validation_report.results %}
+{{ validation_result.severity }}:
+Breached policy: {{ validation_result.source_policy.name }}
+{{ validation_result.source_policy.description }}
+{% if debug %}Debug: {{ validation_result.message }}{% endif %}
+{% endfor %}
+{%- endif -%}