Finalize broker-related CLI interface before release

[GeigerJ2]

Considerations:

• verdi profile command to switch broker
• ZMQ as the default; if something breaks, echo the command to switch to RMQ
• Even keep any option for not using a broker (is there any use case for this anymore?)

Ping @mbercx

Current CLI on main (at e3aae79b107d4b90443db961016460dc806f9d49):

~/a/a/g/aiida-core main ❯ verdi presto --help

Usage: verdi presto [OPTIONS]

  ...

  When the `--use-zmq` flag is toggled, the command skips the RabbitMQ auto-
  detection and directly configures the ZMQ broker. To switch to RabbitMQ
  later, use `verdi profile configure-rabbitmq`.

Options:
  --use-postgres                  When toggled on, the profile uses a
                                  PostgreSQL database instead of an SQLite
                                  one. The connection details to the
                                  PostgreSQL server can be configured with the
                                  relevant options. The command attempts to
                                  automatically create a user and database to
                                  use for the profile, but this can fail
                                  depending on the configuration of the
                                  server.
  --use-zmq                       When toggled on, the profile uses the ZMQ
                                  broker, which requires no external services
                                  and is started automatically with the
                                  daemon. When not specified, the command
                                  automatically tries RabbitMQ first and falls
                                  back to ZMQ if unavailable. To switch to
                                  RabbitMQ later, use `verdi profile
                                  configure-rabbitmq`.
  --no-broker                     When toggled on, no message broker is
                                  configured. This means the daemon cannot be
                                  started and processes cannot be submitted.
                                  Useful for profiles used only for data
                                  exploration and querying.
...

~/a/a/g/aiida-core main ❯ verdi profile --help

Usage: verdi profile [OPTIONS] COMMAND [ARGS]...

  Inspect and manage the configured profiles.

Options:
  -v, --verbosity [notset|debug|info|report|warning|error|critical]
                                  Set the verbosity of the output.
  -h, --help                      Show this message and exit.

Commands:
  configure-rabbitmq  Configure RabbitMQ for a profile.
  delete              Delete one or more profiles.
  ...

[mbercx]

verdi profile command to switch broker

So e.g. replace verdi profile configure-rabbitmq -> verdi profile set-broker. Only runs when daemon is stopped & there are no active processes?

ZMQ as the default; if something breaks, echo the command to switch to RMQ

Not sure we can echo when anything breaks. We should warn during verdi presto, as is being done now. but make the warnings more visible (if that hasn't happened yet).

Note: if it isn't clear: the current default when RMQ is reachable is still RMQ, --use-zmq must be used to make ZMQ the default atm in this case. I suppose we can add a --use-rmq instead. This is pretty aggressive, and a breaking change in behaviour. I feel if someone is using RMQ and happy, I'm not sure if we should push ZMQ on them.

Even keep any option for not using a broker (is there any use case for this anymore?)

I think a no-broker solution still allows running calculations without any daemon, which the ZMQ doesn't: you have to start the daemon to start the broker. Most users would need to submit processes, not run them. But I think keeping the no-broker is still fine. Also: until we declare ZMQ stable we should keep no-broker as a fallback for users that don't want to use RMQ.