Make sure that an amaru node always reconnects to its upstream peer

[etorreborre]

Abstract

At the moment an amaru node will fail to reconnect to an upstream node if that one stops then restarts.
We need to write a test showing that a node will always reconnect when the upstream node has temporarily been disconnected.

Why?

We need operations to be reliable, so that SPOs don't need to restart the node manually in that situation.

How?

This seems to be a bug somewhere since the current Manager code in amaru-network is supposed to schedule a reconnection.

Testing Strategy / Acceptance Criteria

We should have a script (or possibly an in-memory test, using TCP over 2 nodes, like this one), for those 2 situations:

1. The initiator node is started, while the responder (upstream) node is down. Then the responder node is started, and the initiator node should connect and synchronize.

2. Both nodes are started and the responder node is restarted. We should observe a reconnection and the synchronisation resuming.

Discussion points

No response

Dependencies & Related Tasks

No response

Checklist

• I understand that feature requests and unrefined work item should be open as GitHub Discussions instead.
• I have assigned this item to an existing milestone from the roadmap
• I have added a label capturing the impact of this item (i.e. value for users/stakeholders if successful)
• I have added a label capturing the delivery risk of this item (i.e. how likely is it that this task will succeed as planned)
• I have added a label capturing the effort of this item (i.e. how large is the task?)

[etorreborre]

Fixed with #653