kolla-ansible 部署OpenStack queens版本笔记 - 好文

<>kolla-ansible 部署OpenStack queens版本笔记

<>一. 实验环境：

* 6台主机安装CentOS7
minimal系统32G内存，1T+500G双硬盘（其中一个为后期ceph部署做准备），三张千兆网卡（一张用于后期ceph（enp3s0f0），另两张网卡分别作为控制网络（enp0s31f6）以及neutron桥接（enp3s0f1）网络）
* 网络规划：
host IP address remark
controller01 10.132.226.51 1
controller02 10.132.226.52 2
controller03 10.132.226.53 3
compute01 10.132.226.54 4
compute02 10.132.226.55 5
kolla 10.132.226.200 6
virtulal IP 10.132.226.70
虚拟地址池 10.132.226.71-99
<>二. 控制以及计算节点初始化操作：

* 使用以下脚本对每个计算机进行初始化配置（kolla除外）执行 sh initnode.sh <http://initnode.sh>
n(n代表第几台主机) # /usr/bin/bash ls -l /etc/sysconfig/network-scripts|awk
'/ifcfg-enp[0-9]*/ {print$9}' > default_name.txt i=5 c=0 cat default_name.txt |
while read line do cd /etc/sysconfig/network-scripts cp $line ${line}.bak sed -i
's/BOOTPROTO=dhcp/BOOTPROTO=static/g' $line sed -i 's/ONBOOT=no/ONBOOT=yes/g'
$line c=$(expr $c + 1) if [ "${c}" == "1" ]; then continue; elif [ "${c}" == "2"
]; then i=$(expr $i + 2) echo -e "\nIPADDR=10.132.226.${i}${1}" >> $line echo
"NETMASK=255.255.255.0" >> $line echo "GATEWAY=10.132.226.254" >> $line echo
"DNS1=192.168.0.1" >> $line echo "DNS2=114.114.114.114" >> $line else sed -i
'1,11d' $line sed -i '1i\OVS_BRIDGE=br-ex' $line sed -i '1i\DEVICETYPE=ovs'
$line sed -i '1i\BOOTPROTO=none' $line sed -i '1i\TYPE=OVSPort' $line fi done
systemctl stop firewalld&& systemctl disable firewalld && systemctl status
firewalld yum update -y yuminstall -y wget vim net-tools wget -P
/etc/yum.repos.d/
https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo yuminstall -y
docker-cemkdir -pv /etc/docker systemctl restart docker && systemctl status
dockerecho -e
"\n10.132.226.200\tkolla\n10.132.226.51\tcontroller01\n10.132.226.52\tcontroller02\n10.132.226.53\tcontroller03\n10.132.226.54\tcompute01\n10.132.226.55\tcompute02"
>> /etc/hosts if [ $1 -lt 4 ]; then echo "controller0${1}" > /etc/hostname else
num=$(expr $1 - 3) echo "compute0${num}" > /etc/hostname fi reboot
*
各节点主机初始化内容：

* 配置网卡信息
* 关闭防火墙
* 安装docker
* 修改hostname以及添加hosts信息
*
设置各节点主机之间免密登录

* 在各主机依次执行 ssh-keygen ssh-copy-id -i ~/.ssh/id_rsa.pub root@controller01
* 将authorized_key文件发放到各主机的~/.ssh/目录 scp authorized_keys kolla:~/.ssh/ scp
authorized_keys controller01:~/.ssh/scp authorized_keys controller02:~/.ssh/ scp
authorized_keys controller03:~/.ssh/scp authorized_keys compute01:~/.ssh/ scp
authorized_keys compute02:~/.ssh/
<>三. kolla主机配置

*
安装docker：

* 从阿里云下载docker的repo文件： [root@kolla ~]# wget -P /etc/yum.repos.d/
https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
* 安装docker-ce： [root@kolla ~]# yum install -y docker-ce
* 配置国内镜像： [root@kolla ~]# mkdir -p /etc/docker [root@kolla ~]# vim
/etc/docker/daemon.json { "registry-mirrors": [
"https://7g5a4z30.mirror.aliyuncs.com"] }
* 启动docker [root@kolla ~]# systemctl daemon-reload && systemctl enable docker
&& systemctl restart docker
* 检查镜像站点配置是否正确 [root@kolla ~]# docker pull hello-world
*
安装依赖软件

* 安装pip并更新 [root@kolla ~]# yum insatll epel-release -y [root@kolla ~]# yum
insatll python-pip -y [root@kolla ~]# pip install -U pip
* 修改pip源 [root@kolla ~]# mkdir ~/.pip [root@kolla ~]# vim ~/.pip/pip.conf [
global] trusted-host = pypi.douban.com index-url = http://pypi.douban.com/simple
* 安装其他依赖包 [root@kolla ~]# yum install python-devel libffi-devel gcc
openssl-devel libselinux-python -y
*
安装配置ansible：

* 先使用pip安装再使用yum安装，可以防止某些py包版本太低 [root@kolla ~]# pip install ansible [
root@kolla ~]# yum install ansible -y
* 在/etc/ansible/ansible.cfg配置文件中添加以下内容： [defaults] host_key_checking=False
pipelining=True forks=100
*
安装配置kolla-ansible：

* 使用pip安装kolla-ansible： pip install kolla-ansible
* 复制global.yml和password.yml文件到/etc/kolla目录: cp -r
/usr/share/kolla-ansible/etc_examples/kolla /etc/kolla/
* 复制all-in-one 和multinode 文件到当前操作目录： cp
/usr/share/kolla-ansible/ansible/inventory/*.
* 修改global.yml文件
global.yml <http://paste.ubuntu.org.cn/4360073>
* 拉取镜像 kolla-ansible pull -vvv
* 再次修改global.yml文件（因为上一个文件拉取的镜像缺少nova-compute等镜像）
global.yml <http://paste.ubuntu.org.cn/4360075>
* 拉取镜像 kolla-ansible pull -vvv
*
上传镜像到本地registry仓库：

* 配置Docker共享挂载： [root@kolla ~]# mkdir -p /etc/systemd/system/docker.service.d
[root@kolla ~]# vim /etc/systemd/system/docker.service.d/kolla.conf [Service]
MountFlags=shared [root@kolla ~]# systemctl daemon-reload && systemctl restart
docker && systemctl status docker
* 启动registry容器，并将端口映射到4000端口 [root@kolla /]# docker run -d --name registry
--restart=always -p 4000:5000 -v /opt/registry:/var/lib/registry registry:2.6.2
* 修改Docker服务配置，信任本地Registry服务 [root@kolla /]# vim
/usr/lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd
--insecure-registry kolla:4000
* 重新启动docker服务 systemctl daemon-reload && systemctl restart docker
* 测试registry服务是否正常： [root@kolla ~]# curl -X GET http://kolla:4000/v2/_catalog
{"repositories":[]}
* 修改镜像tag： for i in `docker images|grep -v registry|grep -v R|awk '{print $1}'
`;do docker image tag $i:queens kolla:4000/$i:queens;done
* push到本地库 for i in `docker images|grep kolla:4000|awk '{print $1}'`;do
docker push$i:queens;done
* 查看镜像是否上传成功： curl -XGET http://kolla:4000/v2/_catalog { "repositories": [
"kolla/centos-source-aodh-api", "kolla/centos-source-aodh-evaluator",
"kolla/centos-source-aodh-listener", "kolla/centos-source-aodh-notifier",
"kolla/centos-source-barbican-api",
"kolla/centos-source-barbican-keystone-listener",
"kolla/centos-source-barbican-worker", "kolla/centos-source-blazar-api",
"kolla/centos-source-blazar-manager", "kolla/centos-source-ceilometer-central",
"kolla/centos-source-ceilometer-compute",
"kolla/centos-source-ceilometer-notification", "kolla/centos-source-ceph-mds",
"kolla/centos-source-ceph-mgr", "kolla/centos-source-ceph-mon",
"kolla/centos-source-ceph-nfs", "kolla/centos-source-ceph-osd",
"kolla/centos-source-ceph-rgw", "kolla/centos-source-chrony",
"kolla/centos-source-cinder-api", "kolla/centos-source-cinder-backup",
"kolla/centos-source-cinder-scheduler", "kolla/centos-source-cinder-volume",
"kolla/centos-source-cloudkitty-api", "kolla/centos-source-cloudkitty-processor"
,"kolla/centos-source-collectd", "kolla/centos-source-congress-api",
"kolla/centos-source-congress-datasource",
"kolla/centos-source-congress-policy-engine", "kolla/centos-source-cron",
"kolla/centos-source-designate-api",
"kolla/centos-source-designate-backend-bind9",
"kolla/centos-source-designate-central", "kolla/centos-source-designate-mdns",
"kolla/centos-source-designate-producer", "kolla/centos-source-designate-sink",
"kolla/centos-source-designate-worker", "kolla/centos-source-dnsmasq",
"kolla/centos-source-elasticsearch", "kolla/centos-source-etcd",
"kolla/centos-source-fluentd", "kolla/centos-source-freezer-api",
"kolla/centos-source-glance-api", "kolla/centos-source-gnocchi-api",
"kolla/centos-source-gnocchi-metricd", "kolla/centos-source-gnocchi-statsd",
"kolla/centos-source-grafana", "kolla/centos-source-haproxy",
"kolla/centos-source-heat-api", "kolla/centos-source-heat-api-cfn",
"kolla/centos-source-heat-engine", "kolla/centos-source-horizon",
"kolla/centos-source-influxdb", "kolla/centos-source-ironic-api",
"kolla/centos-source-ironic-conductor", "kolla/centos-source-ironic-inspector",
"kolla/centos-source-ironic-pxe", "kolla/centos-source-iscsid",
"kolla/centos-source-karbor-api", "kolla/centos-source-karbor-operationengine",
"kolla/centos-source-karbor-protection", "kolla/centos-source-keepalived",
"kolla/centos-source-keystone", "kolla/centos-source-kibana",
"kolla/centos-source-kolla-toolbox", "kolla/centos-source-kuryr-libnetwork",
"kolla/centos-source-magnum-api", "kolla/centos-source-magnum-conductor",
"kolla/centos-source-manila-api", "kolla/centos-source-manila-data",
"kolla/centos-source-manila-scheduler", "kolla/centos-source-manila-share",
"kolla/centos-source-mariadb", "kolla/centos-source-memcached",
"kolla/centos-source-mistral-api", "kolla/centos-source-mistral-engine",
"kolla/centos-source-mistral-executor", "kolla/centos-source-mongodb",
"kolla/centos-source-multipathd", "kolla/centos-source-murano-api",
"kolla/centos-source-murano-engine", "kolla/centos-source-neutron-bgp-dragent",
"kolla/centos-source-neutron-dhcp-agent", "kolla/centos-source-neutron-l3-agent"
,"kolla/centos-source-neutron-lbaas-agent",
"kolla/centos-source-neutron-metadata-agent",
"kolla/centos-source-neutron-openvswitch-agent",
"kolla/centos-source-neutron-server",
"kolla/centos-source-neutron-server-opendaylight",
"kolla/centos-source-neutron-sriov-agent",
"kolla/centos-source-neutron-vpnaas-agent", "kolla/centos-source-nova-api",
"kolla/centos-source-nova-compute", "kolla/centos-source-nova-compute-ironic",
"kolla/centos-source-nova-conductor", "kolla/centos-source-nova-consoleauth",
"kolla/centos-source-nova-libvirt", "kolla/centos-source-nova-novncproxy",
"kolla/centos-source-nova-placement-api", "kolla/centos-source-nova-scheduler"]
}
*
修改部署配置文件

* 修改当前目录下的multinode文件：mutinode <http://paste.ubuntu.org.cn/4360082>
* 修改/etc/kolla/global.yml文件：global.yml <http://paste.ubuntu.org.cn/4360343>
*
部署：

* 生产随机密码文件： kolla-genpwd
* 修改horizon登录界面admin密码： [root@kolla ~]# vim /etc/kolla/passwords.yml
keepalived_password: mFbTVxF6XyrrT8NqaN5UpFB098GEXuZ9oQyfQI14
keystone_admin_password: admin# 更改此处 keystone_database_password:
C4EzIx0zhoFjsG9dA9TBRaZfbFIdT3f9sCe7jGyg
* 引导配置各节点依赖软件: kolla-ansible -i ./multinode bootstrap-servers PLAY RECAP
*************************************************************************************************************************************************************
compute01: ok=38 changed=7 unreachable=0 failed=0 compute02 : ok=38 changed=7
unreachable=0 failed=0 controller01 : ok=38 changed=7 unreachable=0 failed=0
controller02: ok=39 changed=17 unreachable=0 failed=0 controller03 : ok=38
changed=7 unreachable=0 failed=0 localhost : ok=1 changed=0 unreachable=0 failed
=0
* 进行预部署检查： kolla-ansible -i ./multinode prechecks PLAY RECAP
************************************************************************************************************************************************************
compute01: ok=26 changed=1 unreachable=0 failed=0 compute02 : ok=26 changed=1
unreachable=0 failed=0 controller01 : ok=91 changed=1 unreachable=0 failed=0
controller02: ok=87 changed=1 unreachable=0 failed=0 controller03 : ok=87
changed=1 unreachable=0 failed=0 localhost : ok=6 changed=1 unreachable=0 failed
=0
* Cinder出现错误 TASK [cinder : Checking LVM volume group exists for Cinder]
***********************************************************************************************************
skipping:[controller01] skipping: [controller02] skipping: [controller03] [
DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `
result|failed` use `result is failed`. This feature will be removed in version
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False
in ansible.cfg. fatal: [compute01]: FAILED! => {"changed": false, "cmd": ["vgs",
"cinder-volumes"], "delta": "0:00:00.009794", "end": "2018-10-13
18:33:13.868282", "failed_when_result": true, "msg": "non-zero return code",
"rc": 5, "start": "2018-10-13 18:33:13.858488", "stderr": " Volume group
\"cinder-volumes\" not found\n Cannot process volume group cinder-volumes",
"stderr_lines": [" Volume group \"cinder-volumes\" not found", " Cannot process
volume group cinder-volumes"], "stdout": "", "stdout_lines": []} [DEPRECATION
WARNING]: Using tests as filters is deprecated. Instead of using `result|failed`
use`result is failed`. This feature will be removed in version 2.9.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg. fatal:[compute02]: FAILED! => {"changed": false, "cmd": ["vgs",
"cinder-volumes"], "delta": "0:00:00.010114", "end": "2018-10-13
18:33:13.860281", "failed_when_result": true, "msg": "non-zero return code",
"rc": 5, "start": "2018-10-13 18:33:13.850167", "stderr": " Volume group
\"cinder-volumes\" not found\n Cannot process volume group cinder-volumes",
"stderr_lines": [" Volume group \"cinder-volumes\" not found", " Cannot process
volume group cinder-volumes"], "stdout": "", "stdout_lines": []} * 解决方案： [
root@compute02 .ssh]# vgdisplay --- Volume group --- VG Name centos System ID
Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG
Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 1 Act PV 1 VG Size<
1.82 TiB PE Size 4.00 MiB Total PE 476806 Alloc PE / Size 476806 /<1.82 TiB
Free PE / Size 0 / 0 VG UUID FEgDXH-SBlh-x29N-qU0f-Wajd-2sJ6-rbUre5[
root@compute02 .ssh]# dd if=/dev/zero of=./disk.img count=200 bs=512MB 200+0
recordsin 200+0 records out 102400000000 bytes (102 GB) copied, 509.072 s, 201
MB/s[root@compute02 .ssh]# losetup -f /dev/loop0 [root@compute02 .ssh]# losetup
/dev/loop0 disk.img [root@compute02 .ssh]# pvcreate /dev/loop0 Physical volume
"/dev/loop0" successfully created. [root@compute02 .ssh]# vgcreate
cinder-volumes /dev/loop0 Volume group "cinder-volumes" successfully created
* 进行实际部署： kolla-ansible -i ./multinode deploy
*
初始化OpenStack

*
删除ipadress的py包并重新安装（版本过低下一步客户端安装会出错，原先安装其他包的时候作为依赖包安装的ipaddress无法通过pip删除并升级，只能手动删除再安装最新版本）：
[root@kolla ~]# cd /usr/lib/python2.7/site-packages/ [root@kolla site-packages]
# rm -rf ipaddress* [root@kolla site-packages]# pip install ipaddress
* 安装OpenStack CLI客户端: [root@kolla site-packages]# pip install
python-openstackclient python-glanceclient python-neutronclient
* 设置环境变量： [root@kolla site-packages]# . /etc/kolla/admin-openrc.sh
* 编辑初始化脚本中的网络配置： [root@kolla ~]# vim /usr/share/kolla-ansible/init-runonce
EXT_NET_CIDR='10.132.226.0/24' EXT_NET_RANGE=
'start=10.132.226.130,end=10.132.226.169' EXT_NET_GATEWAY='10.132.226.254'
* 执行初始化脚本： [root@kolla ~]# . /usr/share/kolla-ansible/init-runonce Checking
for locally available cirros image. None found, downloading cirros image. %
Total % Received % Xferd Average Speed Time Time Time Current Dload Upload
Total Spent Left Speed 100 12.1M 100 12.1M 0 0 2040k 0 0:00:06 0:00:06 --:--:--
2716k Creating glance image. ······ Done. To deploy a demo instance, run:
openstack server create \ --image cirros \ --flavor m1.tiny \ --key-name mykey
\ --nic net-id=89a1f674-e89f-4e6d-b96d-2875446adc1e \ demo1

热门工具换一换