Skip to content

Presentation 2025‐06‐12

Jump to bottom
Stéphane Brunner edited this page Jun 11, 2025 · 1 revision

Automating CI/CD for Geospatial Projects with GitHub Apps

Context

Historically, we used the c2c-bot-gis-ci and c2c-bot-gis-ci-2 accounts for CI/CD automation in our geospatial projects. Unfortunately, the first account was banned by GitHub. This experience highlighted the significant risks associated with relying on personal or bot accounts that require broad permissions across repositories.

To address these issues, I developed a dedicated GitHub application with the following key goals:

  • Avoid potential account restrictions by using an official GitHub App instead of a user account
  • Enhance security by eliminating the need for CI tokens with excessive permissions
  • Be easier to manage, don't need to copy the workflow files to each repository ans each stabilization branch

Modules

Changelog

Generates a categorized changelog based on pull requests, and includes commits not attached to any pull request.

Example result: https://github.com/camptocamp/c2cwsgiutils/releases/tag/6.1.0 Used Milestone: https://github.com/camptocamp/c2cwsgiutils/milestone/30

Backport

Documentation: https://github.com/camptocamp/github-app-geo-project/wiki/Module-%E2%80%90-Backport Manages labels and adds a failed workflow status if a backport encounters conflicts.

Pull Request Checks

Checks spelling in pull request titles and commit messages. Prevents merge commits and enforces best practices.

Pull Request Links

Automatically adds a link to Jira based on the branch name. Customizable results, e.g.: https://github.com/camptocamp/ngeo/pull/9780

Outdated Comments

When Copilot review is run multiple times, old review comments can become spammy. This module hides outdated review results.

Example: https://github.com/camptocamp/github-app-geo-project/pull/945

Delete Old Workflow Jobs

Primarily used to clean up the workflow list by deleting old workflow jobs. Example: https://github.com/camptocamp/c2cgeoportal/actions

Patch

Applies patches to pull requests based on a diff provided in an artifact.

Versions

Collects dependency versions from all stabilization branches to create forward and backward compatibility tables. Result: https://geoservices-int.camptocamp.com/github/dashboard/versions?repository=camptocamp/c2cciutils

Audit

Creates pull requests for:

Also provides a dashboard for unfixable CVEs: https://geoservices-int.camptocamp.com/github/dashboard/audit

Workflow (experimental)

Lists failed workflows. Example result: https://geoservices-int.camptocamp.com/github/dashboard/workflow

Clean (experimental)

Designed to delete artifacts related to a pull request, such as:

  • Docker images
  • Folders in the gh-pages branch

Auto Review/Merge/Close (legacy, unused)

Applies rules to automatically review, merge, or close pull requests. This module is now considered legacy.

Dispatch Publish (legacy, unused)

Forwards a repository dispatch event from one repository to another, enabling cross-repository dispatches without requiring a CI token.

Technical Information

The application is split into two main components: the web frontend and a worker (two separate deployments).

Communication between these components occurs exclusively through a PostgreSQL database, which primarily functions as a message queue.

The worker is built as an asyncio application, providing superior memory efficiency compared to traditional thread or process-based approaches.

Technology Stack

  • Backend: Python with asyncio
  • Database: PostgreSQL (as a message queue)
  • Authentication: GitHub OAuth2
  • Deployment: Docker containers
  • Architecture: Microservices (frontend + worker)
  • API: GitHub webhooks and REST API integration
  • Configuration: YAML-based profiles with inheritance

Modularity

The application is highly modular, supports:

  • Multiple applications
  • Separate module lists per GitHub application
  • Configuration profiles with inheritance capabilities
  • Per-GitHub project customization options
  • Seamless scaling with multiple workers
  • Support for multiple workers from different Docker images
  • Comprehensive configuration options

If you're interested in using this application for your specific requirements, please feel free to contact me for implementation details.

Documentation

For detailed documentation on each module, check the project's Wiki.

Status

Currently, my project shouldn't needs anymore a token in the CI/CD workflow files, The gopass secrets are still used to be able to publish the application on Docker hub.

Clone this wiki locally
⚡