@apollo/gateway: Make ApolloGateway.stop() reliable and required

[glasser]

The (not particularly documented) ApolloGateway.stop() method didn't reliably
stop the gateway from polling. All it did was cancel a timeout. But if it is
called while pollServices is in the middle of running, it would do nothing, and
then pollServices would carry on and set another timeout.

Better semantics would be for stop() to reliably stop polling: allow the current
poll to complete, ensure that there will be no more polls, and then (async)
return. This PR implements those semantics, by implementing an explicit state
machine for ApolloGateway's polling.

One reason that these bugs were able to survive is that calling stop() is often
unnecessary. In apollographql/apollo-server#3105 we
chose to unref the polling timeout to allow the Node process to exit if it's
the only thing left on the event loop, instead of encouraging users of
ApolloGateway to be responsible and call stop() themselves. While this may
be reasonable when the gateway's lifecycle is merely a schema polling timer, we
may have future changes to the gateway where proper lifecycle handling is more
important. So this PR also moves away from the world where it's fine to not
bother to explicitly stop the gateway.

That said, in the common case, we don't want users to have to write gateway
stopping code. It makes more sense for stopping the ApolloGateway to be the
responsibility of ApolloServer.stop(), just as ApolloServer.stop() can
trigger events in plugins. So in the recently-released Apollo Server v2.20,
ApolloServer.stop() calls ApolloGateway.stop(). This should mean that in
most cases, the missing unref shouldn't keep the process running, as long as
you've run ApolloServer.stop() (and if you don't, it's likely that other parts
of the server are keeping your process running).

What if you're still running an old version of Apollo Server? For a bit of
backwards compatibility, ApolloGateway detects if it appears to be connected
to Apollo Server older than v2.18. If so, it continues to do the old
unref(). If you're using v2.18 or v2.19, the upgrade to v2.20 should be pretty
painless (it's mostly just minor bugfixes). If you're using ApolloGateway on its
own for some reason, and this change causes your processes to hang on shutdown,
adding a stop() call should be pretty straightforward. (If we learn that this
change really is devastating, we can always go back to an unconditional
timer.unref() later.)

Fixes apollographql/apollo-server#4428.

[glasser]

Before merging this I should implement the Apollo Server change described in it and release v2.20. It's ready for review but I'll mark as draft for this reason.

[trevor-scheer]

Should I be able to call load again after awaiting the stop?

[trevor-scheer]

Probably disregard, I can tell below it's explicitly written with the intention of not allowing that.

[trevor-scheer]

Cool! TIL

[glasser]

FWIW I actually don't know offhand if you get unbounded stack sizes from recursion in async functions.

[glasser]

(I checked and you do.)