6

Background

last week we had a major outage, we identified the bottle neck to be redis number of connections: enter image description here

Our quick fix was to scale redis from cache.m4.4xlarge to chache.m4.10xlarge (vertical scaling)

However this scaling is quite expensive: enter image description here

we would like to scale horizontally: enter image description here Unfortunately, the application code we are using now only allows us to either create a redis cluster with multiple shards, or using a single shard with multiple replicas (see discussion here).

Question

considering that our bottle neck is the connection limit, would it be getter to scale by clustering and increasing the shard number, or by having a single shard and replicating it?

4
  • 2
    Does this help? – 030 Dec 31 '19 at 20:20
  • What did you end up doing? Article about Partitioning may help as well. – Kyslik Feb 16 '20 at 20:22
  • 2
    We found out that our application code was very inefficient. we optimized and redis stopped being a bottle neck – abbood Feb 16 '20 at 20:26
  • software has lots of bottlenecks so starting there is always a good step. – Max N Feb 1 at 21:50
0

Redis Cluster could be considered as an OOTB solution for the Scalability requirements. There is another interesting open source library known as Redis Shard for sharding implementations on Redis. Please find the GitHub repository here.

0

Just for the sake of completeness: Redis documentation https://redislabs.com/ebook/part-3-next-steps/chapter-10-scaling-redis/ suggest a general rule of thumb - add replicas for reads, add shards for writes and memory capacity. And it does make sense.

The replica can have a complete copy of its shard. The more shards the smaller they are. Also writing to one shard does not limit other ones.

If the only concern is reading, then replicas are the simplest solution.

If you need both, just combine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.