How to deploy kafka in docker

1. Build Docker
2. Enter the container
3. Modify the configuration file
4. Test Kafka

1. Build Docker

Here I use docker-compose deployment directly, so you need to install compose in advance.
Since you are going to use compose, the yml file is naturally indispensable.

First, create a new directory and create a new yml file in the directory

insert image description here

The contents of the file are as follows:

version: '2'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    volumes:
      - ./data:/data
    ports:
      - "2181:2181"
       
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME:127.0.0.1
      KAFKA_MESSAGE_MAX_BYTES: 2000000
      KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1:compact"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./kafka-logs:/kafka
      - /var/run/docker.sock:/var/run/docker.sock
 
  kafka-manager:
    image: sheepkiller/kafka-manager
    ports:
      - 9020:9000
    environment:
      ZK_HOSTS: zookeeper:2181

Then use this yml file to start our project

$ docker-compose up -d

You can see that three new containers have been created.

insert image description here

2. Enter the container

We enter the interactive mode of the kafka container with the following command

$ docker exec -it kafkademo01_kafka_1 /bin/bash

Because the higher version of Kafka has built-in Zookeeper, we do not need to enter the Zookeeper container. Therefore, the deployment of zookeeper in the yml file can be omitted.

Then enter the root directory of kafka

$ cd /opt/kafka

3. Modify the configuration file

$ cd /config

insert image description here

The first thing to modify is the zookeeper configuration file: zookeeper.properties
(Comments deleted)

dataDir=/opt/kafka/zooLogs
clientPort=2182
maxClientCnxns=0
admin.enableServer=false

Then modify the kafka configuration file: server.porperties
(Comments deleted)

############################## Server Basics ##############################                                                                                                                                                                                                                                                                                                             
broker.id=0                                                                                                                                        
############################## Socket Server Settings ##############################                                                                                                                                                                                                                                                                                              
listeners=PLAINTEXT://127.0.0.1:9093

############################## Socket Server Settings ##############################                                                                      
listeners=PLAINTEXT://127.0.0.1:9093                                                                                                                    
num.network.threads=3                                                                                                                                   
num.io.threads=8                                                                                                                                        
socket.send.buffer.bytes=102400                                                                                                                         
socket.receive.buffer.bytes=102400
                                                          
socket.request.max.bytes=104857600                                                                                                                      
############################### Log Basics ##############################                                                                                  
log.dirs=/opt/kafka/kafkaLogs                                                                                                                           
num.partitions=1                                                                                                                                        
num.recovery.threads.per.data.dir=1                                                                                                                     
############################## Internal Topic Settings #############################                                                                                                   
offsets.topic.replication.factor=1                                                                                                                       
transaction.state.log.replication.factor=1                                                                                                               
transaction.state.log.min.isr=1                                                                                                                         
############################### Log Retention Policy #############################                                                                        
log.retention.hours=168                                                                                                                                                                                                                                               
log.segment.bytes=1073741824                                                                                                                            
log.retention.check.interval.ms=300000                                                                                                                  
############################# Zookeeper #############################                                                                                   
zookeeper.connect=127.0.0.1:2182                                                                                                       
zookeeper.connection.timeout.ms=18000                                                                                                                   
############################### Group Coordinator Settings ###############################                                                                  
group.initial.rebalance.delay.ms=0                                                                                                                      
port=9093                                                                                                                                                
advertised.host.name=127.0.0.1                                                                                                                           
message.max.bytes=2000000                                                                                                                                
advertised.port=9093

4. Test Kafka

Here are some basic commands:

Start ZooKeeper

zookeeper-server-start.sh ../config/zookeeper.properties

Start Kafka

kafka-server-start.sh ../config/server.properties

Create a Theme

kafka-topics.sh --create --zookeeper 127.0.0.1:2182 --replication-factor 1 --partitions 1 --topic test

View created topics

kafka-topics.sh --list --zookeeper 127.0.0.1:2182

Producer

kafka-console-producer.sh --broker-list 127.0.0.1:9093 --topic test

consumer

kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9093 --topic test --from-beginning

This is the end of this article about the steps to deploy Kafka with Docker. For more information about deploying Kafka with Docker, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in: