1, Concepts and programs

This article only introduces the fast construction scheme , No gossiping . Construction plan , as follows :




Zookeeper independent 3 Station cluster ( Expand according to the project situation ), One for leader, Others are follower.


Kafka Multi node and multi node broker Cluster scheme of ( Expand according to the project situation ), Not used kafka Self contained zookeeper.




2, Deployment preparation

JDK http://www.oracle.com/technetwork/java/javase/downloads/index.html
<http://www.oracle.com/technetwork/java/javase/downloads/index.html>


Zookeeper https://www.apache.org/dyn/closer.cgi/zookeeper/
<http://www.oracle.com/technetwork/java/javase/downloads/index.html>


Kafka http://kafka.apache.org/downloads <http://kafka.apache.org/downloads>


because ZK,Kakfa Operation depends on JVM Environmental Science , Need to be installed first JDK( There are many on the Internet , No more description )


in addition , The above middleware is installed uniformly /home catalog , This directory is the default in the following scheme introduction .




3, Cluster deployment

3.1 Zookeeper

1, Unzip package :


tar -xzvf zookeeper-3.4.11.tar.gz 2, Enter profile directory ,cd
zookeeper-3.4.11/conf, newly build zoo.cfg. Each cluster node uses the same configuration .

3, establish zookeeper-data folder ( And zk Installation directory peer ), And create a new file in this folder myid(linux command :touchmyid), The content is 1. according to zoo.cfg in server.x=ip:port1:port2 Of x value , Setting up different cluster nodes myidde value . as :192.168.0.71 node zookeeper Of myid The content is 1,192.168.0.72 node zookeeper Of myid The content is 2,192.168.0.73 node zookeeper Of myid The content is 3, To ensure the uniqueness in the cluster .




4, start-up Zookeeper service , Execute the command in the installation directory of each node :



./bin/zkServer.sh start


5, Check for proper operation , Execute the command in the installation directory of each point :

./bin/zkServer.sh status



Error contactingservice? Don't panic , Because other nodes have not started yet , Unable to connect problem , Start all services of other cluster nodes . Check again :




among 1 Set as Mode:leader,2 Set as Mode:follower.

Related problems , Viewable bin Directory zookeeper.out Output log , Further investigation .

above ,Zookeeper Cluster deployment completed .



3.2 Kafka

1, Unzip package


tar -xzvf kafka_2.12-1.0.0.tgz
2, Enter profile directory , cdkafka_2.12-1.0.0/config/, use server. properties copy 2 file :server1.
properties,server2.properties. among ,broker.id Ensure uniqueness within the cluster .


Server1.properties Join configuration ( Before deleting a commented configuration # Symbol ):



broker.id=1 listeners=PLAINTEXT://192.168.0.63:9092
advertised.listeners=PLAINTEXT://192.168.0.63:9092 log.dirs=/home/kafka-logs1
Server2.properties Join configuration ( Before deleting a commented configuration # Symbol ):



broker.id=2 listeners=PLAINTEXT://192.168.0.63:9093
advertised.listeners=PLAINTEXT://192.168.0.63:9093 log.dirs=/home/kafka-logs2
modify Zookepeer to configure (Kafka Default to self-contained Zookeeper to configure ):

zookeeper.connect=192.168.0.71:2181,192.168.0.72:2181,192.168.0.74:2181
3, start-up Kafka Server:


bin/kafka-server-start.sh config/server1.properties &
bin/kafka-server-start.sh config/server2.properties &
above ,Kafka Cluster deployment completed