I have autoscaling groups that I want to dynamically scale up/down based on the amount of lag showing in certain Kafka topics. There's a 1:1 relationship between topic and autoscaling group that needs to be scaled. I'm fairly sure I can't use CloudWatch metrics to approach this.
Is there a way I could accomplish this using Jenkins without creating race conditions and other issues?
Note: My application is not constrained by CPU, Disk I/O, or any other metric provided by CloudWatch; the constraint is the application's ability to process from incoming Kafka topics.