We're planning to expand our software to the South East Asia region. A self-hosted Mongo Cluster is fully set up at China's AWS Data Center. How can we set up the MongoDB replicate set to AWS Singapore and allow writing on both regions and rely on MongoDB for asynchronously sync up data behind the scenes?
At this point, it is not possible to have that exactly.
- You can enable async replication from one MongoDB instance to several read replicas.
- You can have two writable instances, and write to the same collections but you can't write to the same documents on both servers. This is achievable by using sharding. But each server will have half of the data.
Since you are in aws, you can implement your own data synchronizer using serverless. It's not a great solution, but it gets the work done.
Let's call the mongdb running on China as mongo-cn and the MongoDB that will run on Singapore mongo-si
- Step 1: enable replication set on mongo-cn, this will not replicate data anywhere, but will create the oplog which will allow you to use change streams
- Step 2: create a fargate container that will have one job and one job only, read the change stream and publish all writes to sqs-si
- Step 3: create a lambda trigger on sqs-si that runs for every sqs message, this lambda will get the sqs message and write it to mongo-si
- Step 4: stop mongo-cn, create an ami from mongo-cn, clear the sqs-si and start the mongo-cn again
- Step 5: copy the mongo-cn snapshot to Singapore and start it as mongo-si
- At this point, mongo-cn is being replicated to mongo-si, now you need to create a second subscriber
- Step 6: create a second fargate subscribing to all changes in the changestrem from mongo-si and publishing those changes to sqs-cn
- Step 7: create a lambta trigger on sqs-cn that runs for every sqs message, this lambda will get the message and write it to mongo-cn
Now you have two writeable mongo db instances being asynchronously replicated across regions, you have buffered writes in sqs and some control/logging by using cloudwatch and lambda triggers.
You will also have zero consistency if the two regions decide to write to the same document at the same time.
This is not a good solution, but some scenarios might require it.
I'm assuming you are using Amazon DocumentDB for the MongoDB cluster.
As of today you can't have active/active configuration and asynchronous replication for DocumentDB. The closest feature for DocumentDB are global clusters which allow global reads, but not global writes.
Moreover, keep in mind that AWS China is entirely separated from the AWS Global Network. Even if there were a feature for active/active configuration, you probably would not be able to set it up between AWS China and other AWS regions.