Connect through a proxy

[delvedor]

There are some use cases where users need to connect to the upstream server via a proxy that is not handling connections transparently.

I would like to add this functionality directly into Undici, and from the user perspective, it would look like this:

const client = new Client({
  url: 'http://localhost:3000',
  proxy: 'http://localhost:3030'
})

Supporting this feature is also useful for working with proxy for debugging purposes, such as mitmproxy.
I've implemented a similar solution for the Node.js's core HTTP client via a custom agent.

What do you think?

[ronag]

We already have Client.connect could we use that somehow?

[delvedor]

I think so, yeah. What I'm suggesting is to add this feature directly into Undici.
The main reason is because this is a real user need and npm is full of HTTP proxy agents for Node.js core. Having it as a first class citizen would help quite a lot.

[ronag]

What I'm suggesting is to add this feature directly into Undici.

I would like to avoid this. That's something for higher level libraries. I'd prefer if we could implement this through composition of lower level API's.

e.g. Client.connect + connect function as argument in to the constructor?

[delvedor]

I would like to avoid this. That's something for higher level libraries.

This is my point. There are not even one of the uselrand HTTP libraries that are implementing this. There are multiple agents implementations and users are left alone figuring out what to do.
Having it directly as part of Undici, will solve this annoying issue, what do you think?

[ronag]

I think the first step is to make it possible by composition using low level API's. Then we can look into adding a higher level API. Maybe as an example (like we are doing here) to begin with.

[delvedor]

Oks, let's try with that :)
I'm trying to figure out how to do this. The TL;DR of what we need is that the socket that the client will use for sending requests, is the one returned by a CONNECT request to the proxy with the correct path (which means that the socket creation needs to be asynchronous).
I did an initial very rough experiment here, but it does not seem to work. Any idea?

[delvedor]

Very likely the reason why the code above is not working is because the internal connect function is also creating a parser, so you can't swap the internal socket.
Probably the easiest solution is to allow the injection of a socket, but creating the socket via connect is an asynchronous process. I'm trying to figure out how hard would be (and if makes sense) to make connect asynchronous.