infrastructure.md

Lantern - Infrastructure

Note

This page is specific to the BAS Catalogue.

Overview

This diagram shows this project's infrastructure components:

Tip

The Processing Scripts component shown in the diagram currently consists of the Non-Interactive Publishing Script.

Environments

Available environments:

• development:
  • for prototyping and making changes (see Development documentation)
  • hosted locally with an optional Local Stack for external infrastructure
• staging:
  • referred to as testing publicly
  • externally accessible
  • for infrastructure testing (i.e. HTTPS configuration, deployment workflows, etc.)
  • for experimentation and previewing content by authors and invited testers
• production:
  • referred to as live publicly
  • externally accessible
  • for general use

Development environments may be created and destroyed as needed. Staging and Production environments are long-lived.

Deployment

• Environment Module
  • managed via Ansible 🛡️
• Non-Interactive Publishing Script
  • managed via Ansible 🛡️

Hosting

This diagram shows this project's hosting components:

Endpoints:

• development: localhost:9000
• staging (testing): data-testing.data.bas.ac.uk, composed of:
  • lantern-testing.data.bas.ac.uk for public content
  • BAS Operations Data Store for Trusted Publishing
• production (live): data.bas.ac.uk, composed of:
  • lantern.data.bas.ac.uk for public content
  • BAS Operations Data Store for Trusted Publishing

The testing and live environments share their endpoints with the legacy Discovery Metadata System (DMS), via reverse proxying. The BAS HAProxy load balancer proxies applicable requests to either:

• a relevant AWS Cloudfront Distribution (for public content)
  • controlled by the data_redirect.txt load balancer config file (🔒)
• or a relevant part of the BAS Operations Data Store 🛡️
  • controlled by the data_internal_redirect.txt load balancer config file (🔒)

Note

For testing the UKRI managed AWS offering, a replica of the production S3 bucket and parallel CloudFront distribution are in use. This parallel CF distribution is used by the BAS Load Balancer for 100% of live traffic.

See WSF/bas-aws#71 🔒 for more information.

Infrastructure as Code

OpenTofu, an open-source fork of the Terraform infrastructure as code tool, is used to manage some project infrastructure in resources/envs/main.tf.

Remote state is managed by the BAS Terraform Remote State 🛡️ project.

To apply this infrastructure:

• install tools (brew install opentofu awscli 1password-cli)
• configure credentials for the BAS AWS 🛡️ account (aws configure)
• copy resources/envs/terraform.tfvars.tpl to resources/envs/terraform.tfvars and populate credentials/values

Then run:

% cd resources/envs
% opentofu init
% opentofu apply

Components

1Password

• Service Account 🔒
  • to allow access to secrets in Continuous Integration
  • managed manually as per Setup documentation

Sentry

• Project 🔒
  • for Error monitoring
  • managed via Infrastructure as Code and manually as per Setup documentation

GitLab

• Project Repository 🛡️
  • Public Mirror
  • managed manually
• Project User 🔒
  • for committing records and interacting with issues
  • managed via Infrastructure as Code
• Records Repository 🔒️
  • GitLab bot user PAT 🔒.
  • for Storing records in GitLab
  • managed via Infrastructure as Code and manually as per Setup documentation
  • requires regular Rotation

Power Automate

• Item Enquires 🔒
  • GitLab bot user PAT 🔒
  • for Item Enquires
  • managed via Infrastructure as Code and manually as per Setup documentation
  • requires regular Rotation

Plausible

• Dashboard 🔒
  • for Web Analytics
  • managed manually as per Setup documentation

Cloudflare

• Turnstile 🔒
  • Site and Secret Keys 🔒
  • for Bot protection
  • managed via Infrastructure as Code and manually as per Setup documentation

Font Awesome

• Icon Kit 🔒
  • for Icons
  • managed manually as per Setup documentation

ArcGIS Online

• OAuth application 🔒
  • for adding AGOL items to records as distribution options and syncing record metadata to AGOL items
  • managed manually as per Setup documentation

Exporters

• AWS S3 publishing buckets & CloudFront distributions:
  • Staging 🔒:
    • IAM user for workstation module 🔒
  • Production 🔒:
    • IAM user for workstation module 🔒
  • for Exporters to publish public content
  • S3 Versioning is enabled on the production bucket to allow recover previous rendered output that cannot be recreated (due to downtime or template changes)
  • DNS records for CloudFront aliases are registered in the relevant AWS Route53 zone
  • TLS certificates for these aliases are managed via AWS Certificate Manager
  • managed via Infrastructure as Code
• Secure hosting:
  • provided by the BAS Operations Data Store 🛡️
  • which in turn uses the BAS LDAP directory for authentication and authorisation
  • All Environments 🔒
    • SSH credentials for workstation module 🔒
  • for Exporters to publish trusted content
  • managed manually as per Setup documentation

Note

For testing the UKRI managed AWS offering, a replica of the production S3 bucket and parallel CloudFront distribution are in use. This parallel CF distribution is used by the BAS Load Balancer for 100% of live traffic.

See WSF/bas-aws#71 🔒 for more information.