Nginx reverse proxy setup issues with Docker containers

Question

I have two running docker containers:

1. A Docker container running Nginx that serves Angular app resource files that were copied into the Docker image on docker build
2. A Docker container that is an api that is current listening on port 5007 internally on the container and 5007 externally on the host machine

I am setting up a staging environment and the following external urls:

• app.staging.mysite.com
• account.staging.mysite.com

app.staging.mysite.com hits port 80 and responds with the Angular app resources no problem.

Now I want to get account.staging.mysite.com external API URL request to hit my API Docker container listening on port 5007. Requests from account.staging.mysite.com will hit port 80 of the host machine.

Nginx will receive the account.staging.mysite.com port 80 requests and proxy them to Docker container 127.0.0.1:5007 while at the same time still serve my Angular app file by default for all external urls/ domains that I do not explicitly express for proxy in my Nginx config.

Instead of serving my Angular app I would like to use Nginx to forward the request to port 5007 so the my Account API can response. So I have altered my Nginx config to the following:

upstream accountstaging{
    server 0.0.0.0:5007
}
server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;

    # Main
    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
        try_files $uri$args $uri$args/ /index.html;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}
server {
    listen 80;
    server_name account.staging.mysite.com;

    location / {
        proxy_pass         http://accountstaging;
        proxy_redirect     off;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Host $server_name;
    }
}

The thing that really surprises me is now there is no default_server hits. In other words, app.staging.mysite.com, which was working and responding with my Angular app resources is no longer working when I added the second server listening on port 80 specifically for account.staging.mysite.com.

So really this question is not that much about Docker containers, this is really about Nginx configuration. Though I am not entirely sure that my Docker containers could be excluded from being part of the problem. So here is my docker-compose.yml for account.staging.mysite.com:

version: '3'

services:
  apistaging:
    build: 
      context: ./
      dockerfile: docker/staging/Dockerfile
    image: tsl.api.account.image
    container_name: tsl.api.account.container
    ports:
      - "5007:5007"
    environment: 
      ASPNETCORE_URLS: http://+:5007

And here is my docker-compose.yml for app.staging.mysite.com:

version: '3'

services:
  frontend:
    build: 
      context: ./
      dockerfile: docker/Dockerfile
    image: tsl.web.frontend.image
    container_name: tsl.web.frontend.container
    ports:
      - "80:80"

Here is my Docker file for Nginx that serves my Angular app and is hopefully going to act as a reverse proxy as well, you can see that Nginx default_server servers my Angular app resource files that I copied into this Docker image on docker build:

FROM centos:7
MAINTAINER Brian Ogden

# Not currently being used but may come in handy
ARG ENVIRONMENT
ENV NODE_VERSION 6.11.1

RUN yum -y update && \
    yum clean all && \
    yum -y install http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm \
    yum -y makecache && \
    yum -y install nginx-1.12.0 wget

# Cleanup some default NGINX configuration files we don’t need
RUN rm /etc/nginx/conf.d/default.conf

#############################################
# NodeJs Install
#############################################

#Download NodeJs package
RUN wget -q -O - https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.gz \
    | tar --strip-components=1 -xzf - -C /usr/local

# https://stackoverflow.com/a/35774741/1258525
# use changes to package.json to force Docker not to use the cache
# when we change our application's nodejs dependencies:
COPY ./package.json /tmp/package.json
RUN cd /tmp && npm install
RUN mkdir /app && cp -a /tmp/node_modules /app/

WORKDIR /app
COPY . /app

RUN npm run build-$ENVIRONMENT

RUN cd /app && cp -a dist/* /usr/share/nginx/html
COPY ./docker/conf/frontend.conf /etc/nginx/conf.d/frontend.conf
COPY ./docker/conf/nginx.conf /etc/nginx/nginx.conf     

EXPOSE 80

CMD ["nginx"]

Is this issue possibly because of 80:80 binding for the Angular app frontend docker container?

Answer 1

Thanks to this question and answer here, I was able realize that I had two issues going on:

1. the containers have different default Docker networks because I am using two different docker-compose.yml files, I had envisioned my Ngnix proxy working independently from any of my API containers entirely, including the docker-compose, more on that issue below
2. the second issue is simply when I tried to proxy to 127.0.0.1:5023 that is localhost inside the Ngnix container, not the network outside of the Nginx proxy container

So the different default networks being created by docker-compose for my Nginx proxy docker container and my api docker container are because I am using two different docker-compose.yml files. This is because I have Jenkins builds for many API microservices so the have independent docker-compose files and I needed a Nginx proxy to forward requests on port 80 to each microservice.

To test this out, created a docker-compose.yml for both containers, the API and the Nginx proxy:

version: '3'

services:
  reverseproxy:
    build: 
      context: ./
      dockerfile: docker/nginxproxy/docker/Dockerfile
    image: tsl.devops.reverseproxy.image
    container_name: tsl.devops.reverseproxy.container
    ports:
      - "80:80"
  apistaging:
    build: 
      context: ./
      dockerfile: docker/staging/Dockerfile
    image: tsl.api.example.image
    container_name: tsl.api.example.container
    ports:
      - "5023:5023"
    environment: 
      ASPNETCORE_URLS: http://+:5023

Yes there was still an issue, the proxy pass to http//:127.0.0.1:5023, that forward remains in the Nginx Docker container and never finds the API running on the Docker host, I simply needed to use the docker-compose.yml service name to get to it:

upstream accountstaging {
    server apistaging:5023;
}

server {

    listen 80;
    server_name account.staging.mysite.com;

    location / {
        proxy_pass         http://accountstaging;
        proxy_redirect     off;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Host $server_name;
    }
}