6

I have two containers (Docker):

  • TensorFlow container
  • NodeJS container used as a frontend to the TensorFlow container.

Both containers have TCP listeners on a port specified using environment variables.

If I want to scale the NodeJS and TensorFlow containers, for example 2 instances of NodeJS and 4 instances of TensorFlow, what are my options for connecting them to evenly spread the load across the 4 instances of TensorFlow?

Currently I'm using the TensorFlow container name inside the NodeJS container, but I'm pretty sure that would only take advantage of a single TensorFlow container... Or??

Improve this question
1

2 Answers 2

Reset to default
1

The classic answer to balancing load is, of course, a load balancer. There are many different methods of doing this, but we're interested in ones that work well with dynamic containers.

This is one of the things an orchestration layer will address. In Kubernetes, for instance, you would create a Tensor Flow service, and scale it to however many pods you need (or use autoscaling!). Kubernetes then provides an ip for the service that kube-proxy load balances across the pods.

There are also service mesh options that replace this layer and perform a similar function.

Improve this answer
-1

You should assign a specific port to any containers.
For this configuration you can use docker-compose file such as following example:

version: '3'

services:
  TensorFlow-1:
      image: gcr.io/tensorflow/tensorflow:latest-gpu
      container_name: first-tensor
      ports:
        - "9001:8888"

  TensorFlow-2:
      image: gcr.io/tensorflow/tensorflow:latest-gpu
      container_name: second-tensor
      ports:
        - "9002:8888"

Then in the docker-compose.yml path, on command-line do it: 

sudo docker-compose up

Or

sudo docker-compose up -d

NOTE: -d option is for run above command in the background.

Finally you have two tensorflow containers with 9001 and 9002 ports.

Improve this answer
5
  • I fail to see how you'd tell the 4 nginx container to spread their loads on then 2 tensor flow ones, and that sound tedious to scale out of this simple example
    – Tensibai
    Commented Jun 21, 2018 at 19:03
  • @Tensibai On this answer, I create two containers of tensorflow as a simple exaplme, you can create four tensorflow container with scale of that, using 9001, 9002, 9003, 9004 ports.
    – Benyamin Jafari
    Commented Jun 21, 2018 at 20:16
  • And what about how to use it in the other containers?
    – Tensibai
    Commented Jun 22, 2018 at 6:20
  • @Tensibai The docker have an IP, and each of containers have an specific IP. Tensorflow container port on this example is 8888 and we have four tensorflow instance (container) with exterior ports (9001, 9002, 9003, 9004). If you want use each of tensorflow container must be use <tesnorflow-containers-IP>:8888 or use <docker-IP>:900X
    – Benyamin Jafari
    Commented Jun 22, 2018 at 19:34
  • 1
    I'll stop trying to make you edit this answer into something usable, you're answering aside of the question which has very few value in my eyes as it's a basic container orchestration problem you won't solve with statically typed names/addresses in the docker file when you're up to 50 containers and more
    – Tensibai
    Commented Jun 22, 2018 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.