An Approach to Multi-Tenant Applications with MongoDB

NoSQL databases like MongoDB allow the ability to scale easily, develop applications faster by supporting flexible schema. However, due to the flexible schema, it’s not the first choice database for most people. I believe that MongoDB can be used innovatively or creatively in order to build multi-tenant applications. This can be accomplished by handling the client metadata in one database, while storing the client specific data in separate databases. Let us explore one such approach below where the application allows users to define their own schema and perform CRUD operations on their schema.

In the application architecture described in Figure 1, we see that if the application is split into one main database to store the metadata for the application users, it’s possible to store the client data in their own separate databases, this allows for us to scale a particular client’s database on demand. For most small companies that provide software as a service, I would speculate that it’s common for them to find themselves with one or two large clients that demand for the architectural changes that benefit them, while affecting the other clients negatively (in poorly designed systems, anyway). The primary advantage in storing the database separately, is that while scaling individual client databases, it’s also possible for us to scale the application by spinning up new instances of the servers, but that’s out of the scope of this article.

app-architecture
Figure1: Multi-Tenant Architecture with Multiple Databases

Storing Client Account Information:

Let’s set up the main database (System DB) to store the account information. The account information contains one user login for the account along with the data schema.

{
    "_id" : ObjectId("557126a6edd785072443982c"),
    "email" : "saichaitanya88@gmail.com",
    "password" : "<password>",
    "lastName" : "lastname",
    "firstName" : "firstname",
    "podName" : "v2gncto",
    "createdAt" : ISODate("2015-06-05T04:33:42.191Z"),
    "updatedAt" : ISODate("2015-06-06T01:44:03.761Z"),
    "otherInfo" : {}
}

The parameter podName contains the database name that contains the client data, the podName parameter can be extended to store more database connection string.

Storing Client Database Schema:

Client specific schema is stored into the CustomObjects collection:

{
    "_id" : ObjectId("558a021a0232718a087ed237"),
    "name" : "Person",
    "customObjectName" : "person",
    "description" : "Person customObject",
    "accountId" : ObjectId("557126a6edd785072443982c"),
    "createdAt" : ISODate("2015-06-24T01:04:26.393Z"),
    "updatedAt" : ISODate("2015-06-24T02:03:50.203Z"),
    "createdBy" : ObjectId("557126a6edd785072443982c"),
    "updatedBy" : ObjectId("557126a6edd785072443982c"),
    "modelDefinition" : [
        {
            "_id" : ObjectId("558a021a0232718a087ed238"),
            "name" : "Created At",
            "fieldName" : "createdAt",
            "description" : "Created At",
            "type" : "Date",
            "scope" : "System",
            "createdAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "updatedAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "createdBy" : ObjectId("557126a6edd785072443982c"),
            "updatedBy" : ObjectId("557126a6edd785072443982c")
        },
        {
            "_id" : ObjectId("558a021a0232718a087ed239"),
            "name" : "Updated At",
            "fieldName" : "updatedAt",
            "description" : "Updated At",
            "type" : "Date",
            "scope" : "System",
            "createdAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "updatedAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "createdBy" : ObjectId("557126a6edd785072443982c"),
            "updatedBy" : ObjectId("557126a6edd785072443982c")
        },
        {
            "_id" : ObjectId("558a021a0232718a087ed23a"),
            "name" : "Created By",
            "fieldName" : "createdBy",
            "description" : "Created By",
            "type" : "ObjectId",
            "scope" : "System",
            "createdAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "updatedAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "createdBy" : ObjectId("557126a6edd785072443982c"),
            "updatedBy" : ObjectId("557126a6edd785072443982c")
        },
        {
            "_id" : ObjectId("558a021a0232718a087ed23b"),
            "name" : "Updated By",
            "fieldName" : "updatedBy",
            "description" : "Updated By",
            "type" : "ObjectId",
            "scope" : "System",
            "createdAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "updatedAt" : ISODate("2015-06-24T01:04:26.393Z"),
            "createdBy" : ObjectId("557126a6edd785072443982c"),
            "updatedBy" : ObjectId("557126a6edd785072443982c")
        },
        {
            "_id" : ObjectId("558a1006e42e219619f5b495"),
            "name" : "DOB",
            "fieldName" : "dOB",
            "description" : "Date Of Birth",
            "type" : "Date",
            "scope" : "Application",
            "createdAt" : ISODate("2015-06-24T02:03:50.203Z"),
            "updatedAt" : ISODate("2015-06-24T02:03:50.203Z"),
            "createdBy" : ObjectId("557126a6edd785072443982c"),
            "updatedBy" : ObjectId("557126a6edd785072443982c")
        }
    ]
}

Now, with the schema describing the CustomObjects collection, their fields and data types, we can build a persistence layer to make sure that the data inserted into the client database complies with the schema defined in the CustomObjects collection. (Persistence layer source code)

Client Custom Data:

With these in place, we can build CRUD actions for any conceivable CustomObject collection that the user can define. The details of this is located in the source code. An example Person record according to the schema defined above:

{
    "_id" : ObjectId("558cada3a1899f635d7d3190"),
    "dOB" : ISODate("2015-06-25T03:15:32.229Z"),
    "createdAt" : ISODate("2015-06-26T01:40:51.394Z"),
    "updatedAt" : ISODate("2015-06-28T17:03:14.143Z"),
    "createdBy" : ObjectId("557126a6edd785072443982c"),
    "updatedBy" : ObjectId("557126a6edd785072443982c")
}

Additional Benefits:

While it’s possible for the users to define their own data schema for CustomObjects, it’s also possible for us to provide a single user interface that allows the users to query and modify the objects. The UI looks at the CustomObject schema, and renders the text boxes (for search and edit) based on the data type of the field.

Search UI
Figure 2: Auto-Generated Search UI
Save UI
Figure 3: Auto-Generated Edit UI

Limitations and Considerations:

One limitation in my preliminary work is that MongoDB did not support Collection Joins until version 3.2, which meant that schemas where multiple collections are linked and referenced may not perform well.

It’s also likely that other databases like RethinkDB, CouchDB, Cassandra might be a better fit for this type of application structure, the database that best fits the needs of the application must be chosen.

Future Work:

With client-side JavaScript becoming ever so popular, it’s possible for us to enhance the database schema and allow storage of JavaScript functions that run on the browser. Taking this one step further, we can even store alternative methods to accommodate similar functionality for modules used by the application clients. For example: Calculation of Income Taxes on wages differs from country to country, if each country is a client, then two separate JavaScript functions can be defined in the schema to allow the application to seamlessly use the appropriate Income Tax Calculation. Another example: Two clients (one car sales company, the other software company) wanting to measure employee performance, the car sales company measures performance based on number of sales made, while the software company measures performance based on the ratio of features to bugs per employee.

The schema can also be extended to contain custom Model validation rules allowing validation rules which depend on other collections to be implemented.

References:

MongoDB – https://www.mongodb.org/
NodeJS – https://nodejs.org/en/
Github Source Code – https://github.com/saichaitanya88/contact_manager.prototype

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s