0

I am facing a strange problem with logging when I set up a network of three docker containers each running a small flask app.

My python application looks as follows:

from flask import Flask, request
import sys
import logging
import logging.handlers
import os

app = Flask(__name__)

logger = logging.getLogger('Logger')
logger.setLevel(logging.DEBUG)
log_handler = logging.StreamHandler(sys.stdout)
logger.addHandler(log_handler)

service_name=""
if 'SERVICE_NAME' in os.environ:
    service_name = os.environ['SERVICE_NAME']

logger = logging.LoggerAdapter(logger, {"serviceName" : service_name})

@app.route("/ping")
def ping():
    logger.info(f'answering ping request')

    return f'{service_name} OK'

if __name__ == "__main__":
    logger.info("Started")

    app.run(debug=True, host='0.0.0.0')

I start three instances of this in a docker network behind nginx:

docker network create tNet

docker run -it -d -p 24224:24224 -v `pwd`:/fluentd/etc -e FLUENTD_CONF=fluentd.conf --name=fluentd_logging fluent/fluentd

docker run -d --log-driver=fluentd --name ms1 -e SERVICE_NAME="ms1" --network tNet testflask
docker run -d --log-driver=fluentd --name ms2 -e SERVICE_NAME="ms2" --network tNet testflask
docker run -d --log-driver=fluentd --name ms3 -e SERVICE_NAME="ms3" --network tNet testflask

docker run -d --name=nginx --network tNet --volume `pwd`/nginx.conf:/etc/nginx/conf.d/default.conf -p 2000:2000 -p 3000:3000 nginx

The nginx configuration is as follows:

upstream app {
    server ms1:5000;
    server ms2:5000;
    server ms3:5000;
}

server {
    listen 2000;

    location / {
        proxy_pass http://app;
    }
}

My fluent configuration looks as follows:

<source>
    @type forward
</source>

<match>
    @type stdout
</match>

When I use curl I can see that each instance of the service is responding:

curl localhost:2000/ping
ms1 OK
curl localhost:2000/ping
ms2 OK
curl localhost:2000/ping
ms3 OK

When I however inspect the logs I can only see logs from the 3rd instance of the service:

docker logs <fluent container ID> | grep -Eis "answering ping"

19-08-06 15:34:46.000000000 +0000 be338a3d5aa0: {"container_id":"be338a3d5aa0b8920366fdf1ba2a267082cee47e681adcea16b6c51c9c82cf0d","container_name":"/ms3","source":"stdout","log":"answering ping request"}

Has anyone an idea? Thank you in advance.

0

I finally found out the problem: the fluent container needs some time to start up; because it's started with --detach the subsequent containers try to connect before fluentd is actually running, only the 3rd instance of the flask application connects when fluentd is already up. So, delaying the startup of the remaining containers solves my problem.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.