Upgrade github-bot server

[richardlau]

Refs: nodejs/github-bot#377 (comment)

Currently the github-bot server is running Node.js 14.16.1 on Debian 8 (Jessie) installed via https://github.com/nodesource/distributions.

build/ansible/roles/github-bot/tasks/main.yml

Lines 16 to 19 in b36beec

	- name: Bootstrap | Add nodesource repo
	apt_repository:
	repo: deb https://deb.nodesource.com/node_14.x jessie main
	state: present

I attempted a quick patch of the playbook to install Node.js 18

diff --git a/ansible/roles/github-bot/tasks/main.yml b/ansible/roles/github-bot/tasks/main.yml
index 7455da3..cda0ea1 100644
--- a/ansible/roles/github-bot/tasks/main.yml
+++ b/ansible/roles/github-bot/tasks/main.yml
@@ -15,7 +15,7 @@

 - name: Bootstrap | Add nodesource repo
   apt_repository:
-    repo: deb https://deb.nodesource.com/node_14.x jessie main
+    repo: deb https://deb.nodesource.com/node_18.x jessie main
     state: present

 - name: Bootstrap | APT Update and upgrade

but this doesn't work

W: Failed to fetch https://deb.nodesource.com/node_18.x/dists/jessie/main/binary-amd64/Packages  HttpError404

we can see that the following URLs do not exist:

• https://deb.nodesource.com/node_18.x/dists/jessie/main/binary-amd64/Packages
• https://deb.nodesource.com/node_16.x/dists/jessie/main/binary-amd64/Packages

so we'll probably need to upgrade the machine to Debian 10 (Debian 8 is long out of support).
e.g. https://deb.nodesource.com/node_18.x/dists/buster/main/binary-amd64/Packages exists.

[targos]

How do you connect to this server? SSH says "debug1: send_pubkey_test: no mutual signature algorithm"

[richardlau]

@targos I have two systems I regularly use that are RHEL 8 and Fedora 37. For the Fedora system it looks like at some point in the past I used update-crypto-policies to use the LEGACY policy, probably to allow connections to some of our build servers on older OSes (like this one -- this would be yet another reason to update it). The alternative, non-system-wide solution, is to modify the .ssh/config entry for the server and add

  PubkeyAcceptedKeyTypes +ssh-rsa

-- just be aware that if you use the write-ssh-config playbook that will overwrite the entry.

[targos]

@richardlau Thanks! It works from my machine running Rocky Linux 8 so I'm going to use that.

[Trott]

Is there anything I can do to help move this forward? We're stuck on updates of dependencies for github-bot because we need to support Node.js 14.x until this upgrade happens.

[targos]

I don't know. We need someone who has the time, knowledge, and infra access to do the changes.

[richardlau]

I don't know. We need someone who has the time, knowledge, and infra access to do the changes.

FTR I haven't done much to the github-bot server (I only gained access to the secrets at the beginning of this year). I've just given access to the "github-bot" area of the secrets repo to everyone in the build-infra team (who already had access to the server via the build-infra ssh key but not all of them had access to the secrets).

Knowledge may be a challenge and involve piecing together what is is secrets and ansible/roles/github-bot (including the bot source code in https://github.com/nodejs/github-bot).

Time is probably the biggest challenge.

[richardlau]

I've made some updates to the ansible/playbooks/create-github-bot.yml playbook and provisioned a new Debian 12 server.
This server is running Node.js 20.15.0 (deployed via the updated playbook). All deployment secrets are decrypted by the playbook from the secrets repo (i.e. running the playbook against the new host was all that was needed to set it up). Being a more modern OS, we no longer need workarounds to allow legacy key types to ssh into it.

I've changed the DNS entry for the bot to the new server, which is currently serving requests and appears to be running as expected (posting comments and job status back to PRs, reploying when https://github.com/nodejs/github-bot is updated). I suggest monitoring for a while (a week?) -- if we encounter any issues we can swap the IP address in CloudFlare back to the old server. Once we're happy there are no issues with the new server we should decomission the old one.

PR: #3803