Switch mongodb databases with sharedb

[kapeskittu]

How can i switch between multiple mongodbs using sharedb?

We are trying to implement a multi tenant application where transactions should be performed only to the database of country from where user belongs to. We maintaing one database for one country on same mongo server. How does this work with sharedb?

[kanwarkakkar]

@kapeskittu I am facing the same issue, is there any resolution?

[ericyhwang]

The ShareDB server doesn't directly support reading/writing to different databases, depending on information in the client messages.

I think you could indirectly achieve that by instantiating one ShareDB Backend per database and then registering incoming client connections to the appropriate Backend instance. That'd be faster and cleaner than having to introspect every single message, too.

A quick illustration based off the Counter example:

const backendMap = new Map();
for (const mongoConfig of mongoConfigs) {
  const db = new ShareDBMongo(mongoConfig.url);
  const backend = new ShareDB({db});
  backendMap.set(mongoConfig.countryCode, backend);
}

  // Register WebSocket connection with appropriate Share Backend for the country
  wss.on('connection', function(ws, req) {
    var stream = new WebSocketJSONStream(ws);
    const countryCode = getUserCountryCode(req.getHeader('Cookie'));  // However you determine country
    const backend = backendMap.get(countryCode);
    if (backend) {
      backend.listen(stream);
    } else {
      // error handling
    }
  });

[kapeskittu]

Hi Kanwar, I was able to imagine such a situation with less minimal code changes using mongos and shard keys of mongodb cluster environment: https://docs.mongodb.com/manual/core/sharded-cluster-query-router/ If it is multitenant application you can introduce country or region of database as shard key for every collection and implement mongos. Other than multiple sharedb instance is another solution as discussed by other user in the forum. Where you will have a gateway app which will route to specific instance of sharedb application based user login and which region he belongs to. Let me know, if you need any other details. Thanks, Krishna Prasad Kattula.

…

On Wed, Apr 3, 2019, 6:16 PM Kanwarjeet singh ***@***.***> wrote: @kapeskittu <https://github.com/kapeskittu> I am facing the same issue, is there any resolution? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#56 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADXPJpJY5VVdR4e33K0N_vc9bbPGnmz7ks5vdKKbgaJpZM4TuhQO> .

[kanwarkakkar]

@ericyhwang isn't shareDb providing multiple DB support?
using

racer.createBackend({
        db: mongo,
        pubsub: pubsub,
        extraDbs: options.extraDbs
      })
    })

While creating racer backend connections?

[alecgibson]

Closing due to inactivity.

[ihsanciftci]

Hello. (Sorry for resurrecting an old issue.)
@ericyhwang

a) If we use url for db parameter, sharedb creates new client for each database. That will be a performance penalty. For that reason you suggest to use case b)
b) If we use function for db parameter, sharedb uses our client to create/open the database.

When we implement according to b) case, for 1000 databases, there will be over 5000 connections that is causing mondodb server to be killed.

**
Is it possible to implement sharedb according to c) case?

c) If sharedb were using our db instance (instead of creating the Db instance from our given client), it would be possible to use the same connection pool for entire application.

Does my suggestion sound nice?
**

Mongodb documents about connection pooling:
https://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html#mongoclient-connection-pooling

"To reduce the number of connection pools created by your application, we recommend calling MongoClient.connect once and reusing the database variable returned by the callback:"

[alecgibson]

@ihsanciftci my first question is why are you running and connecting to 1000 databases? That sounds like a lot.

My second question is if you need to have that many databases, is connecting to them from a single server a sensible and scalable approach? This sounds like you may need to run more backends?

To your question, providing a DB instance might work, but would have at least these issues:

• maintaining backwards compatibility with older mongodb versions could be tricky
• I can't quite remember, but this might also cause issues with creating a client for query polling?

At any rate, if this is an issue you want to persue, you're better off opening a new issue please.

[ihsanciftci]

Thanks for your reply. @alecgibson
I'm not good at NodeJS but I tried the case (c).
There may be better options.

    if (isLegacyMongoClient(client)) {
        self.mongo = self._mongoClient = client;
    }
    else if(client.s) {
        self.mongo = client;
        self._mongoClient = client.s.client;
        self._dbInstance = true;
    }
    else {
        self.mongo = client.db();
        self._mongoClient = client;
    }

ShareDbMongo.prototype.close = function(callback) {
  if (!callback) {
    callback = function(err) {
      if (err) throw err;
    };
  }
  var self = this;
  this.getDbs(function(err) {
    // Ignore "already closed"
    if (err && err.code === 5101) return callback();
    if (err) return callback(err);
    self.closed = true;
    if(!self._dbInstance) {
      self._mongoClient.close(function(err) {
        if (err) return callback(err);
        if (!self._mongoPollClient) return callback();
        self._mongoPollClient.close(callback);
      });
    }

  });
};

I'm creating 4000 sharedb instances at a rate of 20 instances/seconds, in that case mongodb creates 150 connections.
So the above code looks like running correctly.

my first question is why are you running and connecting to 1000 databases? That sounds like a lot.
In fact, this is one of approaches for a multi-tenant system. Another one is using prefix for collections.
One db per tenant approach provides better isolation over collection approach: less complex security operations etc.

I think mongodb and sharedb can handle over ten thousand databases. I think it depends on websocket performance.

This sounds like you may need to run more backends?

Yes. For example if my one backend handles 10 thousands sharedb instances. 10 server scales to 100 000 sharedb instances.

One MongoDB connection pool can work across multiple databases over a mongodb server. I think this is a modern connection pool.
On the other hand, Postgres community does not have such a connection pool. Each database connection has a separate connection pool. This makes implementing "one database per tenant" approach very hard.

I think ShareDB missed that part of connection reusing. This will enable using Sharedb in SaaS.

• I can't quite remember, but this might also cause issues with creating a client for query polling?

I did not try the query polling.