Skip to content

README.dev.md

Latest commit

 

History

History
267 lines (177 loc) · 7.1 KB

README.dev.md

File metadata and controls

267 lines (177 loc) · 7.1 KB

DashVERSE Developer Notes

Requirements

  • OpenTofu (1.6+) or Terraform (1.6+)
  • kubectl (1.28+)
  • helm (3.0+)
  • minikube (1.30+)
  • Docker or Podman
  • Ansible (2.9+)

If you have Nix installed, all dependencies are provided via nix develop.

Links for the requirements

Python

https://www.python.org/downloads

Pyenv (optional)

Pyenv allows developers to install multiple versions of Python distribution and easy switching between the installed versions.

Website: https://github.com/pyenv/pyenv?tab=readme-ov-file#installation

Poetry (optional)

Poetry is used for dependency management of the Python packages.

https://python-poetry.org/docs/#installation

Podman

https://podman.io/docs/installation

Docker

https://docs.docker.com/engine/install

minikube

https://minikube.sigs.k8s.io/docs/start

helm

https://helm.sh/docs/intro/install

Deployment configurations

The deployment settings for both local (testing) and production environments can be found in terraform/environments folder.

Deployment

Quick Start

  1. Start minikube

Note: If you already have a kubernetes cluster, you can skip this step.

minikube start --cpus='4' --memory='4g'

  1. Deploy

    make deploy ENV=local

  2. Verify pods are running

    make status

  3. Access services

On a separate terminal do port forwarding to be able to access the service

make port-forward

  1. Deploy preconfigured dashboards

    make setup-dashboards ENV=local

Then open:

Credentials

Service credentials are auto-generated during deployment and stored securely in Kubernetes secrets. To retrieve them:

./scripts/show-access.sh

This displays:

  • PostgreSQL connection details
  • Superset admin login
  • JWT secret for token signing

You can also retrieve individual credentials with kubectl:

# PostgreSQL password
kubectl get secret dashverse-secrets -n dashverse -o jsonpath='{.data.postgres-password}' | base64 -d

# Superset admin password
kubectl get secret dashverse-secrets -n dashverse -o jsonpath='{.data.superset-admin-password}' | base64 -d

Makefile Targets

Target Description
make deploy Build images and deploy all services
make destroy Remove all services
make status Show deployment status
make port-forward Forward ports to localhost
make logs Tail all service logs
make logs-auth Tail auth service logs
make sync Download EVERSE indicators/dimensions
make sync-apply Download and import to database
make sync-trigger Trigger sync cronjob manually
make jwt Generate JWT token (CLI)
make build-auth Build auth-service image
make setup-dashboards Configure Superset dashboards via Ansible
make seed-data Import sample software and assessments

Manual Deployment

cd terraform
tofu init
tofu apply -var-file="environments/local.tfvars"

Production Deployment

# Deploy all services (builds images and applies Terraform)
make deploy ENV=production

# Populate data
make sync-apply
make seed-data

# Configure Superset dashboards
make setup-dashboards ENV=production

The production configuration (terraform/environments/production.tfvars) includes settings for external URLs used in iframe embedding.

Sync EVERSE Data

Indicators and dimensions are synced from the EVERSE repository: https://github.com/EVERSE-ResearchSoftware/indicators

The sync runs automatically daily at 2am via a CronJob. To trigger manually:

make sync-trigger

Or to sync outside the cluster:

make sync-apply

Authentication

The Auth Service provides a web interface for user registration and JWT token generation.

  1. Open http://localhost:8000 (after port-forward)
  2. Register a new account
  3. Login and generate an API token
  4. Use the token for PostgREST write access

Alternatively, generate a token via CLI:

make jwt

API Documentation

Interactive API documentation is provided using Scalar:

The documentation is automatically generated from OpenAPI specifications and includes an interactive request builder.

Dashboard Configuration

After deployment, configure Superset with pre-built dashboards using Ansible:

make setup-dashboards

This creates five role-based dashboards based on RSQKit roles:

Prerequisites:

  • Ansible (2.9+)
  • Port forwarding running (make port-forward)
  • Superset accessible at localhost:8088

The Superset admin password is automatically retrieved from Kubernetes secrets during setup.

Sample Data

To populate the system with sample software and assessments for testing:

make seed-data

This fetches software metadata from the EVERSE TechRadar repository and generates sample assessments. The data will appear in the Superset dashboards.

Documentation

  • docs/README.dev.md – Developer Notes: Instructions on configuration and deployment
  • docs/architecture/README.md – System Architecture: Detailed diagrams of the system, data flow, and security model.
  • docs/Deployment.md – deployment checklist and prerequisites.
  • docs/Kubernetes.md – operational commands for managing the Minikube deployment.
  • docs/Database.md – details of the PostgreSQL schema, assessment mapping, and populate script usage.
  • docs/API_examples.md – practical PostgREST calls, including the multi-step workflow for creating assessments.

Clean up

Remove all deployed resources:

make destroy

Delete the minikube cluster:

minikube stop
minikube delete
⚡