We are trying to move towards canary deployments.
We want to be able to spin up an entirely new deployment and be able to access the DB, but the DB is privately connected to the existing VPC for security.
How do other people connect the new VPC to the existing DB? Peering? How do you handle the peering connection switching because the subnet would need to change for each deployment. Where do you manage the state of this?
We can't migrate DB to the new VPC because it is TB of data and wouldn't be effective to do this multiple times a day if we get faster at releasing.
I tried to use the same VPC and subnets for multiple deployments, but the load balancer only allows for 1 listener on 1 port, I can't have multiples, so the terraform crashes.
If I try to use individual subnets inside the same VPC, they also crash and there's no easy way to allocate network address space using terraform before the deployment.
We are primarily using ecs but use other services too.