Last month - in Jan 2019, AWS announced that Amazon DocumentDB (with MongoDB compatibility) was Generally available !!
“Amazon DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. Developers can use the same MongoDB application code, drivers, and tools to run, manage, and scale workloads on Amazon DocumentDB and enjoy improved performance, scalability, and availability without having to worry about managing the underlying infrastructure. Customers can easily migrate their MongoDB databases that are on-premises or on Amazon EC2 to Amazon DocumentDB for free (for six months per instance) with virtually no downtime using the AWS Database Migration Service (DMS). There are no up-front investments required to use Amazon DocumentDB, and customers only pay for the capacity they use.”
Having played around with MongoDB in the past, I wanted to give DocumentDB a quick try. Being a developer, I was not content with just provisioning a DocumentDB cluster, and trying out the Mongo shell, but rather wanted to test, if existing applications using MongoDB will work without any changes to code. Of course, things like connection string and other configuration’s will need to be modified ;-)
The key features that I liked about DocumentDB are (please check the official documentation for the latest updates) :
Each instance can range from 2 vCPU/16 GiB to 64 vCPU/488 GiB
Can go up to 16 instances (one primary to 15 replicas) — distributed across Subnets/Availability Zones
Runs within a VPC
Like RDS, we can specify a cluster parameter group that defines configuration settings to be applied to the instances in the cluster.
Data is encrypted at rest, by default ..
Backup retention between 1 and 35 days in which you can perform a point-in-time restore. Automated backups are stored in Amazon S3, which is designed for 99.999999999% durability.
I am as lazy as any other developer, I scouted for some sample apps — where I could test the above hypothesis quickly. Hence, I searched for NodeJS apps which especially uses the very popular Mongoose to talk to MongoDB .. I found a simple CRUD app to talk to a MongoDB at https://codeburst.io/writing-a-crud-app-with-node-js-and-mongodb-e0827cbbdafb. Yes, I can see some eyes rolling that this is a toy app, but the concepts from this blog should serve you well even in your application ;-)
Step 1 — Provision a Amazon DocumentDB cluster
The important details of how the Amazon DocumentDB works are given at https://docs.aws.amazon.com/documentdb/latest/developerguide/how-it-works.html . This was the most easiest part, I quickly spun up a cluster in N.Virginia region. The most important thing to note that the cluster runs in an VPC and the apps which connect to the cluster should probably, in most cases, should also be within the VPC. This makes eminent sense from a security perspective, having read about vulnerable MongoDB databases in the past. Details of accessing the DocumentDB from outside the VPC are given at https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-from-outside-a-vpc.html
To test the cluster, I created a test EC2 instance, and installed mongo shell and tested the connectivity .. The steps are given at https://docs.aws.amazon.com/documentdb/latest/developerguide/getting-started.connect.html
Connect to this cluster with the mongo shell
mongo — ssl — host amzn-docdb-jan2019-mani1.cluster-xxxxx.us-east-1.docdb.amazonaws.com:27017 — sslCAFile rds-combined-ca-bundle.pem — username docdbadmin — password <insertYourPassword>Connect to this cluster with an application
mongodb://docdbadmin:<insertYourPassword>@amzn-docdb-jan2019-mani1.cluster-xxxxx.us-east-1.docdb.amazonaws.com:27017/?ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0
Note, that the connection to DocumentDB is via SSL and we are using native MongoDB tools like the mongo shell to connect to the cluster.
Before you can connect using TLS, you first need to download the public key for Amazon DocumentDB. You can download the public key at the following URL:https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem
You can also disable TLS, the instructions are given at https://docs.aws.amazon.com/documentdb/latest/developerguide/security.encryption.ssl.html — However, I would like to use TLS and encrypt my data in transit between my app and DocumentDB.
Step 2 — Show me the code !!
The first step was to clone the github link — https://github.com/Eslamunto/ProductsApp
I made some very minor changes to app.js, which are highlighted in bold, to read the .pem file and to pass on this options during the time of connection
// app.js
const fs = require(‘fs’);
const certFileBuf = fs.readFileSync(‘./rds-combined-ca-bundle.pem’);var express = require(‘express’);
var bodyParser = require(‘body-parser’);var product = require(‘./routes/product’); // Imports routes for the products
var app = express();// Set up mongoose connection
var mongoose = require(‘mongoose’);
var dev_db_url = ‘mongodb://docdbadmin:password@amzn-docdb-feb2019-mani2.cluster-xxxxxx.us-east-1.docdb.amazonaws.com:27017/productdb?ssl=true&replicaSet=rs0’;
var mongoDB = process.env.MONGODB_URI || dev_db_url;
var options = {
sslCA: certFileBuf
};mongoose.connect(mongoDB,options);
mongoose.Promise = global.Promise;
var db = mongoose.connection;
db.on(‘error’, console.error.bind(console, ‘MongoDB connection error:’));app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: false}));
app.use(‘/products’, product);var port = 1234;
app.listen(port, () => {
console.log(‘Server is up and running on port numner ‘ + port);
});
Start the app and listen to port 1234
node app.js
Step 3 — Testing
You can make REST calls using curl or a tool like POSTMAN, have a look at my test cases ..
Create a record using POST
curl -H “Content-Type: application/x-www-form-urlencoded” -X POST -d “name=mango fruit&price=887” http://localhost:1234/products/create
verify, if the data has been inserted into DocumentDB using the mongo shell and also via a GET call
curl localhost:1234/products/5c672570c5de5542c1f6b818
{“_id”:”5c672570c5de5542c1f6b818",”name”:”mango fruit”,”price”:887,”__v”:0}rs0:PRIMARY> show dbs
admin 0.000GB
productdb 0.000GB
productstutorial 0.000GB
test 0.000GB
rs0:PRIMARY> use productdb
switched to db productdb
rs0:PRIMARY> db.products.find()
{ “_id” : ObjectId(“5c672570c5de5542c1f6b818”), “name” : “mango fruit”, “price” : 887, “__v” : 0 }
rs0:PRIMARY> exit
bye
Update the data using PUT:
curl -H “Content-Type: application/x-www-form-urlencoded” -X PUT -d “name=mango fruit&price=5555” http://localhost:1234/products/5c672570c5de5542c1f6b818/update
Verify using mongo shell
rs0:PRIMARY> use productdb
switched to db productdb
rs0:PRIMARY> db.products.find()
{ “_id” : ObjectId(“5c672570c5de5542c1f6b818”), “name” : “mango fruit”, “price” : 5555, “__v” : 0 }
rs0:PRIMARY>
That’s it, hope this blog gives you an idea on how to communicate between your app and DocumentDB. I guess, we can apply the same principles when our code runs in AWS Lambda or a Docker container ..
Amazon Database migration service can also help you with migrating from MongoDB to DocumentDB — https://docs.aws.amazon.com/dms/latest/userguide/target.docdb.tutorial.html
Finally, please go through the DocumentDB FAQ — https://aws.amazon.com/documentdb/faqs/ — most of the commonly asked questions are answered here. Hope you have a good ride, in your migrations to DocumentDB or creating brand new cloud native apps !!
Bye for now ..
