I'm using a statefulset + headless service to stand up horizontally-scaled dedicated servers. I'm looking for a solution that will allow external traffic to be directed to a specific server and hit it with some URL like https://dedicated-server.my-domain.com/my-pod-1/some/path, though routing in the host itself is fine too (https://my-pod-1.dedicated-server.my-domain.com/some/path)

The current solution is to create a NodePort service with a "Local" externalTrafficPolicy and pod anti-affinity set so no two pods can end up on the same node. This works fine, but that anti-affinity means we must scale our nodes 1-1 with our pods which is both ugly and expensive. Additionally, this means our external URL has a high-numbered port which is breaking under some edge cases I can't mention here.

I was hoping that there might exist some Ingress-based solution to route a host to a service without having to create the specific Ingress object for that host (since the dedicated servers are hooked up to an HPA, they can scale in and out dynamically and there's nothing there to write the Ingresses), but I can't find any documentation for such behavior on nginx-ingress or Istio. The ideal solution allows for connection over port 443 as expected.

  • Two possibilities I've considered are 1. approach the product team and ask that the dedicated server integrates with k8s to create its own Ingress on init, deleting it on shutdown. 2. write a controller that watches for these pods to be created/destroyed and create/destroy an ingress object alongside it. I'd prefer to avoid both solutions, the former because our product team is generally not kubernetes-aware and our application runs outside of k8s as well and the latter because I've never written a controller before so it'll be a lot of effort.
    – Adam Smith
    Dec 23 '21 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.