1

if i have filebeat -> logstash -> elasticsearch pipeline with TLS or HTTPS encryption , is it possible to load balance between 2 destination elasticsearch clusters in active-standby ?
what i mean is, in case one of the clusters goes down i would manually or automatically switch all the data streams to the other cluster.

im looking at various proxy setups but im not sure i understand the terms correctly, and TLS would not solve the problem.

  • That's unclear what you're after here.. Can you explain more what is in each cluster? – Tensibai Feb 22 '18 at 5:34
  • 1
    the situation is that currently we ship log data with logstash to a cloud based ELK . so we have the certificates of the destination installed on the client side . what i want is to be able to switch to another ELK cluster located in another cloud by changing the DNS cname . – Saar Grin Feb 22 '18 at 10:01
  • 1
    And you don't own the certificate to set it up on the other cluster ? – Tensibai Feb 22 '18 at 10:07
  • so basically we now ship data to a commercial ELK SaaS from windows/linux machines with logstash i need to be able to switch over to another ELK SaaS provider without having to reconfigure the logstash or OS on which logstash is installed the idea is to set up a permanent cname on our domain that will currently forward to our current vendor but may be switched over to another vendor the problem is the system is using TLS transport so if the destination hostname doesnt match it wont work, therefore im looking for proxy that is able to transparently load-balance the TLS connection – Saar Grin Feb 26 '18 at 9:32
  • Or just ask your SaaS provider to install a certificate you'll be giving matching your cname... – Tensibai Feb 26 '18 at 12:27
1

You can do this a couple of ways, but I have never seen anyone do this with a CNAME.

One option, if you have control over your IP addressing is to use BGP to fail-over (or load balance) an IP to your cloud hosting providers.

A second option is to invest in a DNS-based geographic load balancing solution, such as the GTM from F5, the GSLB solution from A10, or the GSLB solution from Kemp (F5 is the most commonly used in the industry). With these solutions you will set your name server (NS1 and NS2) for your domain to be the GSLB solution of your choice. You will set a very low TTL for your DNS records and configure health monitors to your ELK stack from the GSLB solution. These can be as complex as logging into ELK and scraping for data or as simple as making sure you can establish a TCP connection and/or ping your server. When this health monitor fails, the GSLB solution will failover traffic to the alternate data center automagically. You will be able to do this as an HA pair with one GSLB solution at once DC as NS1 and the other at your secondary site as NS2. The GSLB will then forward lookup to your previous nameservers which will publish IP addresses for both sites. These will be rewritten to eliminate the IP that the GSLB does not want to direct traffic to at the time (again, this can be used for active-active sites - you can even use geolocation to send users in Europe to a Eurpoean cloud provider and US based users to a US based provider, or do a simple active/standby)

A final way to do this might be with local load balancing solution, such as F5's LTM, A10's SLB, or Kemp's solution. You may also be able to use nginx for this for a cheap solution. In this scenario, you might have this solution onsite in an office with the URL to the load balancer served up by your local DNS server to a local IP (a Virtual IP) on your load balancer. You would then configure the load balance to direct users to the desired cloud provider by configuring a health monitor (of whatever complexity). You would then need to configure this load balancer to bridge your SSL connection. You would add your certificate and private key to the load balancing solution. Users would establish an encrypted connection to the load balancer. This encryption would be stripped off by the load balancer and a new encrypted connection established to the cloud provider by the load balancer, and the data forwarded along over the new SSL tunnel.

  • i have considered many of the options you suggested , but terminating SSL at the LB and sending it on unencrypted is against company policy so not an option essentially im looking for something that can transparently proxy TLS connections assuming i have access to certificates of both endpoints – Saar Grin Feb 26 '18 at 9:29
  • 1
    Load balancing at tcp level works, you don't need access to endpoint certificates unless you're setting up a man in the middle terminating the SSL flow. Moreover, you can be terminating TLs at the load balancer and sending to backend in TLS also. – Tensibai Feb 26 '18 at 12:28
  • @SaarGrin - Yes, you misunderstand. This solution bridges the SSL conn and the data is encrypted on both sides of the reverse proxy. It does not terminate (offload) SSL encryption. On an F5 at least this can also be done without re-encapsulating the traffic in a SSL using SSL. The client establishes an SSL session with the back-end server and the ADC (Application Delivery Controller or load balancer) has the private key and uses it to decrypt and inspect the traffic for the purpose of session persistence. – James Shewey Feb 26 '18 at 16:20
  • @Tensibai - Load balancing at a TCP level won't work if he needs session persistence. Otherwise, you may be logged in to server A, and then your next request goes to server B, which makes you log in again - and may even destroy the login cookie you had for Server A. While you can maintain persistence using the SSL session ID, it is unreliable as session ID are rotated periodically. – James Shewey Feb 26 '18 at 16:24
  • @JamesShewey round robin is not the only load balancing method, ip source hash, ip source/port hash are also valid solution and allow to keep sessions. In case of elasticsearch, each request is a standalone one, so there's nothing problematic to push datas to Elasticsearch with a RR load balancing. – Tensibai Feb 26 '18 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.