How to use Docker-compose to build an ELK cluster

How to use Docker-compose to build an ELK cluster

All the orchestration files and configuration files can be accessed on my Github. You can quickly build a 3-instance ES cluster and a Kibana instance by simply modifying the path in the configuration file.

planning

It is planned to create three ES instances to form a cluster, and create a Kibana instance to connect to the cluster. Each ES instance uses a local configuration file to facilitate the storage and version management of the configuration file. Kibana's configuration file is also placed locally and mounted into the container through file mapping.

The overall directory structure is as follows:

$tree
.
├── docker-compose.yml
├── kibana.yml
├── node1
│ └── es1.yml
├── node2
│ └── es2.yml
└── node3
 └── es3.yml

3 directories, 5 files

Arrangement File

The main orchestration file is docker-compose.yml

version: "2.1"
services:
 es-node1:
 image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
 hostname: es-node1
 expose: #Do not expose the port to applications outside the container - "9001"
 ports: #Expose ports to the host - "9200:9200"
 - "9300:9300"
 volumes:
 - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/node1/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml
 environment:
 - cluster.name=es-cluster
 - bootstrap.memory_lock=true
 - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
 ulimits:
 memlock:
 soft: -1
 hard: -1
 networks:
 es-cluster-network:
 ipv4_address: 172.21.0.10
 es-node2:
 image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
 hostname: es-node2
 expose: #Do not expose the port to applications outside the container - "9002"
 ports: #Expose the port to the host - "9201:9201"
 - "9301:9301"
 volumes:
 - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/node2/es2.yml:/usr/share/elasticsearch/config/elasticsearch.yml
 environment:
 - cluster.name=es-cluster
 - bootstrap.memory_lock=true
 - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
 ulimits:
 memlock:
 soft: -1
 hard: -1
 networks:
 es-cluster-network:
 ipv4_address: 172.21.0.11
 es-node3:
 image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
 hostname: es-node3
 expose: #Do not expose the port to applications outside the container - "9003"
 ports: #Expose the port to the host - "9202:9202"
 - "9302:9302"
 volumes:
 - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/node3/es3.yml:/usr/share/elasticsearch/config/elasticsearch.yml
 environment:
 - cluster.name=es-cluster
 - bootstrap.memory_lock=true
 - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
 ulimits:
 memlock:
 soft: -1
 hard: -1
 networks:
 es-cluster-network:
 ipv4_address: 172.21.0.12
 kibana:
 image: docker.elastic.co/kibana/kibana:6.7.0
 ports:
 - "5601:5601"
 volumes:
 - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/kibana.yml:/usr/share/kibana/config/kibana.yml
 environment:
 -ELASTICSEARCH_URL=http://es-node1:9200
 networks:
 - es-cluster-network
networks:
 es-cluster-network:
 driver: bridge
 ipam:
 driver: default
 config:
 - subnet: 172.21.0.0/16
 gateway: 172.21.0.1

The ES configuration file only selects one example as follows:

cluster.name: elasticsearch-cluster
node.name: es-node1
network.bind_host: 0.0.0.0
network.publish_host: 172.21.0.10
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true 
discovery.zen.ping.unicast.hosts: ["172.21.0.10:9300","172.21.0.11:9301","172.21.0.12:9302"]
discovery.zen.minimum_master_nodes: 2

Kibana's configuration file is as follows

server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://es-node1:9200", "http://es-node2:9201", "http://es-node3:9202" ]
xpack.monitoring.ui.container.elasticsearch.enabled: false

Startup Command

After the configuration file is prepared, you can start the cluster

$ docker-compose up -d

The startup process may be slow. You can see the cluster nodes through the command line.

$ curl http://localhost:9200/_cat/nodes
172.21.0.12 51 96 29 6.53 6.43 3.72 md - es-node3
172.21.0.11 47 96 30 6.53 6.43 3.72 mdi - es-node2
172.21.0.10 49 96 30 6.53 6.43 3.72 mdi * es-node1

In the subsequent use process, you can start and stop the service through the docker-compose command. If you do not want to keep the related instances, you can use docker-compose down to shut down and delete the container.

References

elasticsearch document
Docker-compose custom network, fixed container IP address
Difference between docker-compose ports and expose

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.

You may also be interested in:
  • Docker Compose one-click ELK deployment method implementation
  • Sample code for deploying ELK using Docker-compose
  • How to build an elk system using docker compose
  • Use Docker Compose to quickly deploy ELK (tested and effective)

<<:  js implements axios limit request queue

>>:  Graphical introduction to the difference between := and = in MySQL

Recommend

A time-consuming troubleshooting process record of a docker error

Table of contents origin Environmental Informatio...

Introduction to JWT Verification Using Nginx and Lua

Table of contents Preface Lua Script nignx.conf c...

MySQL 5.6 zip package installation tutorial detailed

Previously, we all used files with the suffix .ms...

Solve the problems that need to be paid attention to when configuring Tomcat's maxPostSize attribute

need: In the near future, we will implement the l...

Detailed explanation of the visualization component using Vue to compare the differences between two sets of data

Table of contents need: Main points: According to...

MySQL 8.0.16 winx64 installation and configuration method graphic tutorial under win10

This article records the specific method of insta...

CSS3 radar scan map sample code

Use CSS3 to achieve cool radar scanning pictures:...

Detailed explanation of this reference in React

Table of contents cause: go through: 1. Construct...

How to install Docker on Windows Server 2016

Recently Microsoft released Windows Server 2016, ...

Example of using CSS3 to achieve shiny font effect when unlocking an Apple phone

0. Introduction August 18, 2016 Today, I noticed ...

Comprehensive explanation of CocosCreator hot update

Table of contents Preface What is Hot Change Coco...

Three properties of javascript objects

Table of contents 1. writable: writable 2. enumer...

Three ways to create a gray effect on website images

I’ve always preferred grayscale images because I t...

Beginner's guide to building a website ⑥: Detailed usage of FlashFXP

Today I will introduce the most basic functions of...

Several important MySQL variables

There are many MySQL variables, some of which are...