1, Concepts and programs

This article only introduces the fast construction scheme, No trouble.. Construction plan, as follows:




Zookeeper Independent3 Taiwan cluster( Expand according to the project situation), One forleader, Others arefollower.


Kafka Multi node and multi nodebroker Cluster scheme of( Expand according to the project situation), No usekafka Self containedzookeeper.




2, Deployment preparation

JDK http://www.oracle.com/technetwork/java/javase/downloads/index.html
<http://www.oracle.com/technetwork/java/javase/downloads/index.html>


Zookeeper https://www.apache.org/dyn/closer.cgi/zookeeper/
<http://www.oracle.com/technetwork/java/javase/downloads/index.html>


Kafka http://kafka.apache.org/downloads <http://kafka.apache.org/downloads>


BecauseZK,Kakfa Operation depends onJVM Environmental Science, Need to be installed firstJDK( Many online, No longer describe)


in addition, The above middleware is installed uniformly/home Catalog, This directory is the default in the following scheme introduction.




3, Cluster deployment

3.1 Zookeeper

1, Unzip package:


tar -xzvf zookeeper-3.4.11.tar.gz 2, Enter profile directory,cd
zookeeper-3.4.11/conf, Newly buildzoo.cfg. Each cluster node uses the same configuration.

3, establishzookeeper-data Folder( Andzk Installation directory peer), And create a new file in this foldermyid(linux command:touchmyid), Content is1. according tozoo.cfg inserver.x=ip:port1:port2 Ofx value, Setting up different cluster nodesmyidde value. as:192.168.0.71 nodezookeeper Ofmyid Content is1,192.168.0.72 nodezookeeper Ofmyid Content is2,192.168.0.73 nodezookeeper Ofmyid Content is3, To ensure the uniqueness in the cluster.




4, start-upZookeeper service, Execute the command in the installation directory of each node:



./bin/zkServer.sh start


5, Check for proper operation, Execute the command in the installation directory of each point:

./bin/zkServer.sh status



Error contactingservice? Mo panic, Because other nodes have not started yet, Unable to connect problem, Start all services of other cluster nodes. Check again:




among1 Taiwan asMode:leader,2 Taiwan asMode:follower.

Related problems, Viewablebin Directoryzookeeper.out Output log, Further investigation.

Above,Zookeeper Cluster deployment completed.



3.2 Kafka

1, Unzip package


tar -xzvf kafka_2.12-1.0.0.tgz
2, Enter profile directory, cdkafka_2.12-1.0.0/config/, Useserver. properties copy2 file:server1.
properties,server2.properties. among,broker.id Ensure uniqueness within the cluster.


Server1.properties Add configuration( Before deleting a commented configuration# Symbol):



broker.id=1 listeners=PLAINTEXT://192.168.0.63:9092
advertised.listeners=PLAINTEXT://192.168.0.63:9092 log.dirs=/home/kafka-logs1
Server2.properties Add configuration( Before deleting a commented configuration# Symbol):



broker.id=2 listeners=PLAINTEXT://192.168.0.63:9093
advertised.listeners=PLAINTEXT://192.168.0.63:9093 log.dirs=/home/kafka-logs2
modifyZookepeer To configure(Kafka Default to self-containedZookeeper To configure):

zookeeper.connect=192.168.0.71:2181,192.168.0.72:2181,192.168.0.74:2181
3, start-upKafka Server:


bin/kafka-server-start.sh config/server1.properties &
bin/kafka-server-start.sh config/server2.properties &
Above,Kafka Cluster deployment completed