Streaming RPCs / GRPC Compatibility

[paralin]

We need to add a API surface for streaming RPCs.

Have a look at how GRPC node does it: http://www.grpc.io/docs/quickstart/node.html

I would recommend mirroring that exactly and then attempting to convert grpc-node to use the protobuf.js pattern.

[dcodeIO]

Once created through Service.create with a RPC implementation, the API actually looks quite similar:

var Greeter = root.lookup("Greeter");
var greeter = Greeter.create(myRpcImpl);

// Now you can use
greeter.sayHello({ name: "you" }, function(err, res) {
  ...
});

The RPC implementation can be quite everything that takes a request and asynchronously returns a response. Signature:

/**
 * RPC implementation passed to {@link Service#create} performing a service request on network level, i.e. by utilizing http requests or websockets.
 * @typedef RPCImpl
 * @function
 * @param {Method} method Reflected method being called
 * @param {Uint8Array} requestData Request data
 * @param {function(?Error, Uint8Array=)} callback Node-style callback called with the error, if any, and the response data
 * @returns {undefined}
 */

[paralin]

Yes, but it needs to be able to take a stream of request or response messages (not sure if you're just comparing the current implementation or saying it doesn't need changing)

…

On Thu, Dec 8, 2016, 6:38 AM Daniel Wirtz ***@***.***> wrote: Once implemented through Service.create <https://github.com/dcodeIO/protobuf.js/blob/master/src/service.js#L159> with a RPC implementation, the API actually looks quite similar: var Greeter = root.lookup("Greeter");var greeter = Greeter.create(myRpcImpl); // Now you can use (rename the method to begin with a lower case letter here?)greeter.SayHello({ name: "you" }, function(err, res) { ... }); The RPC implementation can be quite everything that takes a request and asynchronously returns a response. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgpZ44sI7GK7Vj8fdM-pY8_cod1422Oks5rF-wpgaJpZM4LHWRP> .

[dcodeIO]

I referred to the API described at http://www.grpc.io/docs/quickstart/node.html - which doesn't seem to cover streams explicitly - but I assumed it is also the API for streaming buffers, which in this case should be possible to be implemented with a rpcImpl that reads from and writes to a stream instead.

I might just be missing something. Do you have another reference or example of the API you propose?

[paralin]

Try this one: http://www.grpc.io/docs/tutorials/basic/node.html

…

On Thu, Dec 8, 2016, 9:10 AM Daniel Wirtz ***@***.***> wrote: I refer to the API described at http://www.grpc.io/docs/quickstart/node.html - which doesn't seem to cover streams explicitly - but I assumed it is also the API for streaming buffers, which in this case should be possible to be implemented with a rpcImpl. I might just be missing something. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgpZ_-ugaFg51eEFZckb_TLKNrfLXQ0ks5rGA_WgaJpZM4LHWRP> .

[dcodeIO]

Hmm, stream classes. I'd rather try to avoid these as this potentially adds a lot of usually-unused code to the library. I'll probably try to use the current rpcImpl stuff first to implement streaming RPC. Requirements: Multiple requests, common callback, right?

[paralin]

Yes, as well as a number of events (end, error, data, status, meta)

…

On Thu, Dec 8, 2016, 9:39 AM Daniel Wirtz ***@***.***> wrote: Hmm, stream classes. I'd rather try to avoid these as this potentially adds a lot of usually-unused code to the library. I'll probably try to use the current rpcImpl stuff first to implement streaming RPC. Requirements: Multiple requests, common callback, right? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgpZ7VfllO43PsJD2u6HKNXMb7V9Y8Oks5rGBaYgaJpZM4LHWRP> .

[dcodeIO]

What about...

• data: callback(null, someMessage)
• error: callback(someError)
• end: callback(null, null)

Plus, to signal end on the calling site, sending a null message.

[paralin]

How would you send multiple messages from the client side? Is this really better than the grpc way?

…

On Thu, Dec 8, 2016, 9:45 AM Daniel Wirtz ***@***.***> wrote: What about... - data: callback(null, someMessage) - error: callback(someError) - end: callback(null, null) And to signal end, also sending a null message. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgpZxGUZrIY0BTOVsiWOhtgXcPt2VUwks5rGBf5gaJpZM4LHWRP> .

[dcodeIO]

From an API perspective, using stream classes will most likely always look and feel better. The alternative I am proposing would simply not require dedicated stream classes at all.

Let me prepare an example.

[dcodeIO]

Example

It's now actually using a minimal event emitter to represent the service.