I'm trying to scale up my push notification infrastructure. We currently use laravel app running on 2 ec2 instances horizontally scaled behind a load balancer, and each instance has a beanstalk queue engine that processes jobs created by laravel (including push notification jobs).
the problem is that this solution is not performant. Sometimes we send a marketing push to all our clients.. numbering in the thousands, and the time range difference between the first push notification and the last can be many hours.
I was considering what infrastructure would be good options. I considered AWS SQS, but it turns out that it's max delay job capacity is 15 minutes. In my business, customers can make pre-orders 20 hours before. So what we currently do is scheduled a delayed job 20 hours from now.
But then some people suggested that the delayed part should only happen at the application level (ie delay(work, timedelta(hours=20))
), and that would allow me to use SQS anyways. Their reasoning is that delaying jobs is a business logic thing, and it's best handled at an application level and not an infrastructure level.
I'm not sure I agree.. and that's why I'm picking AWS elastic cache with redis. What do you guys think?