Solution for multiple Docker containers not having the same port number

Background

In Docker, four containers are created using the same image. The network is in bridge mode. Service A uses the same port number (6000) in all four containers. To reduce the number of ports exposed to the outside, nginx is used as a proxy for the four service instances. The four service instances belong to four upstreams, and paths like /service1 and /service2 are used to access the four instances.

At this time, if you access any service locally, a 502 error will be reported, which is puzzling.

connect() failed (111: Connection refused) while connecting to upstream

Compose file

version: '2'
networks:
 nn:
  driver: bridge
services:
 service-1:
  container_name: service-1
  image: foo
  networks:
   -nn
  volumes:
   - ./logs/1:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/1.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000
  
 service-2:
  container_name: service-2
  image: foo
  networks:
   -nn
  volumes:
   - ./logs/2:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/2.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000
 
 service-3:
  container_name: service-3
  image: foo
  networks:
   -nn
  volumes:
   - ./logs/3:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/3.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000
 
 service-4:
  container_name: service-4
  image: foo
  networks:
   -nn
  volumes:
   - ./logs/4:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/4.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000  
 
 nginx:
  container_name: nginx
  image: nginx:1.15-alpine
  ports:
   - 6001:6001
  networks:
   -nn
  volumes:
   - ./nginx/nginx.conf:/etc/nginx/nginx.conf
   - ./logs/nginx:/var/log/nginx

nginx.conf

worker_processes 8;
worker_rlimit_nofile 65535; 
events {
    use epoll;
    worker_connections 65535;
 }
 
http {
    include mime.types;
    default_type aplication/octet-stream;
    sendfile on;
    log_format main '[$time_local]$remote_addr-$upstream_addr "$request" $status $body_bytes_sent';
 
    upstream service1.local {
      server service-1:6000;
    }
    upstream service2.local {
     server service-2:6000;
    }
    upstream service3.local {
      server service-3:6000;
    }
    upstream service4.local {
      server service-4:6000;
    }
 
    server {
      listen 6001;
      client_max_body_size 100M;
      proxy_set_header Host $host:$server_port;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
      location /service1/ {
        proxy_pass http://service1.local/;
      }
      location /service2/ {
        proxy_pass http://service2.local/;
      }
      location /service3/ {
        proxy_pass http://service3.local/;
      }
      location /service4/ {
        proxy_pass http://service4.local/;
      }
      location /nginx_status {
        stub_status on;
        access_log off;
      }
    }
}

At this time, curl localhost:6001/service1/api/v1/.... will report the above 502 error. It stands to reason that each container has its own network card, and the port numbers of different containers should not conflict.

Solution

There is no better solution for now, so we can only use different port numbers for the four services and modify nginx accordingly.

Supplement: Deploy multiple Docker containers on the same server, port redirection issue

In the production environment, deploy multiple containers and access multiple ports;

For example: -p 80:80 -p 81:81

When address 81 exits, directly access the address of port 80.

Misconception: I initially thought it was a cookie problem because I refreshed the cookie (cookies do not distinguish between port numbers)

Finally found the reason: redirect problem, because the logout redirects to the login page

Solution: Configure nginx parameters

proxy_set_header HOST $host; becomes proxy_set_header HOST $host:81;

Because no matter what, the request parameter carries the port number.

There is another method on the Internet: modify the proxy_redirect parameter (but it didn't work after trying it)

The above is my personal experience. I hope it can give you a reference. I also hope that you will support 123WORDPRESS.COM. If there are any mistakes or incomplete considerations, please feel free to correct me.

You may also be interested in:

Solve the problem of specifying udp port number in docker
Raspberry Pi series: Using Docker to install Qinglong Panel and change the port number configuration issues

<<: In-depth analysis of MySQL 8.0 redo log

>>: Detailed explanation and classic interview questions of Vue life cycle and hook functions