Load balancing while deployment

Question

How does a L4 or L7 load balancer knows how to route traffic during deployments. Say, In Canary deployment strategy, how does LB route only 10% traffic to a certain server or in case of Blue-Green deployment strategy, how does LB knows that the updated application is available on another set of servers and I should be routing the traffic to them.

Where should we make those configuration ideally ?

Answer 1

Your load balancer usually offers this as configuration. Using HAProxy as an example (this is similar for most load balancers) you can change:

1. Whether a server is in balance or not
2. What proportion of traffic it should receive
3. What strategy to use to divide traffic between servers

Common use cases for this would be:

• Taking a server out of balance (and allowing existing connections to drain) before running a deployment, then warming it back up with a slower rate of traffic.
• Splitting a small proportion of traffic for a canary release, which you might also use "session pinning" for, to make sure a canary user stays on the canary server, rather than flip-flopping between the old and new version

With HAProxy, you can perform some operations via the command line to take servers in and out of balance. For the load balancing strategy, you'd normally change the config and reload it (which, again, you could automate via the command line).

It's not uncommon for people to set up HAProxy with "redispatch" configured. They can then surprise it by deploying to a server without telling it. HAProxy will receive the failure and re-send the request to a different server. The user has no idea there was any issue. You need to make sure this doesn't cause a domino effect in your scenario.

Nginx is pretty similar. Other load balancers offer things like REST APIs for these operations.