Openstack Install Guide(Ussuri - Ubuntu 18.04) - networking 사용

Ussuri Version - MariaDB

• mysql Ver 15.1 Distrib 10.3.32-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

Docker version

• Docker version 20.10.11, build dea9396

 

1. 물리 환경 및 네트워크 구성

1.1 물리 환경

• 4 Network Interface(External, MGMT, DATA, STORAGE)
• 8GB Main Memory
• 40GB Disk Space

Openstack kolla-ansible Version 10.4.0(Ussuri)

OS	Ubuntu 18.04.5
NIC	External, MGMT, DATA, STORAGE

 

1.2 네트워크 설정

1) root 계정 활성화

# root 계정 암호 설정 및 로그인 $ sudo passwd root

 

2) 방화벽 중지

$ ufw disable $ ufw status Status: inactive

 

3) ipv6 중지

$ vim /etc/sysctl.conf net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1

 

4) SSH 키 분배

$ ssh-keygen -t rsa $ ssh-copy-id root@hosts

 

5) OVS 설치 및 설정

$ apt install openvswitch-switch $ systemctl status openvswitch-switch $ vim /lib/systemd/system/ovsdb-server.service ### Unit 기존 주석 및 신규 Unit 추가 ### 이유 : 재부팅시 OVS 브릿지 보다 일반 네트워크가 먼저 올라와 브릿지 인터페이스가 시작되지 않음 #[Unit] #Description=Open vSwitch Database Unit #After=syslog.target network-pre.target #Before=network.target network.service #ReloadPropagatedFrom=openvswitch-switch.service #PartOf=openvswitch-switch.service [Unit] Description=Open vSwitch Database Unit After=syslog.target network-pre.target dpkg.service local-fs.target Before=network.target network.service PartOf=openvswitch-switch.service DefaultDependencies=no [Service] LimitNOFILE=1048576 Type=forking Restart=on-failure EnvironmentFile=-/etc/default/openvswitch-switch ExecStart=/usr/share/openvswitch/scripts/ovs-ctl \\ --no-ovs-vswitchd --no-monitor --system-id=random \\ start $OVS_CTL_OPTS ExecStop=/usr/share/openvswitch/scripts/ovs-ctl --no-ovs-vswitchd stop ExecReload=/usr/share/openvswitch/scripts/ovs-ctl --no-ovs-vswitchd \\ --no-monitor restart $OVS_CTL_OPTS RuntimeDirectory=openvswitch RuntimeDirectoryMode=0755

 

6) NAS 연결 패키지 설치 및 시간 설정

$ apt-get install nfs-kernel-server $ mkdir -p /etc/kolla/config/nfs_shares $ vim /etc/kolla/config/nfs_shares NASIP:/NFS/os/cinder $ timedatectl set-timezone 'Asia/Seoul'

 

7) networking 서비스 사용을 위한 패키지 설치

$ apt-get install ifupdown $ vim /etc/network/interfaces  ### Interface 확인 및 브릿지 설정 auto lo iface lo inet loopback ##################### External 네트워크 세팅  ##################### auto br-ex allow-ovs br-ex iface br-ex inet static address 192.168.0.X ~ 192.168.0.XX netmask 255.255.255.0 gateway 192.168.0.X dns-nameservers 8.8.8.8 ovs_type OVSBridge ovs_ports enp26s0f1 auto enp26s0f1 allow-br-ex enp26s0f1 iface enp26s0f1 inet manual ovs_bridge br-ex ovs_type OVSPort ##################### DATA 네트워크 세팅 - 사설망 ##################### auto br-int allow-ovs br-int iface br-int inet static address 172.22.0.1 ~ 172.22.0.12 netmask 255.255.255.0 ovs_type OVSBridge ovs_ports br-int auto enp24s0f0 allow-br-int enp24s0f0 iface enp24s0f0 inet static address 172.22.0.101 ~ 172.22.0.112 netmask 255.255.255.0 ovs_bridge br-int ovs_type OVSPort ##################### MGMT 네트워크 세팅 - 사설망 ##################### auto eno1 iface eno1 inet static address 172.21.0.1 ~ 172.21.0.12 netmask 255.255.255.0 ##################### STORAGE 네트워크 세팅 - 사설망 ##################### auto enp26s0f0 iface enp26s0f0 inet static address 172.18.0.1 ~ 172.18.0.12

 

8) modprobe 설정

$ modprobe ip_vs $ modprobe ip6_tables $ vi /etc/modules-load.d/ip_vs.conf ip_vs $ vi /etc/modules-load.d/ip6_tables.conf ip6_tables

 

9) 파이썬 버전 설정

### 파이썬 3버전 설정 $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1 $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2 $ update-alternatives --config python

 

2. Kolla 설치 준비 & 설치

2.1 Kolla 의존성 설치

$ apt-get install python3-dev libffi-dev gcc libssl-dev $ pip3 install ‘ansible<2.10’ $ pip3 install ansible==2.9.6 $ apt install ansible $ vim /etc/ansible/ansible.cnf [defaults] forks          = 100 host_key_checking = False [ssh_connection] pipelining = True

 

2.2 Kolla 설치

• Kolla-ansible 설치시 Openstack Release 버전에 따라 설치한다. 이 문서는 Ussuri 버전에 맞추어 설치한다.
• https://releases.openstack.org/teams/kolla.htm

$ pip install 'kolla-ansible==10.4.0' --ignore-installed PyYAML $ cp -r /usr/local/share/kolla-ansible/etc_examples/kolla /etc/ $ cp -r /usr/local/share/kolla-ansible/ansible/inventory/ /root/

 

2.3 Kolla 설정 파일 세팅(1)

<컨트롤노드 호스트명> network_interface=<내부물리NIC이름> neutron_external_interface=<외부물리NIC이름> kolla_external_vip_interface=<외부물리NIC이름> <컴퓨트노드 호스트명> network_interface=<내부물리NIC이름> api_interface=<관리(없는 경우 내부 물리)NIC이름> storage_interface=<내부물리NIC이름> tunnel_interface=<내부물리NIC이름> $ cd /root/inventory $ vim /root/inventory/multinode # 그 외 설치 패키지에 따라 세팅 변경 [control] controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 [network] controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 [compute] compute01 network_interface=enp24s0f0 api_interface=eno1 storage_interface=enp26s0f0 tunnel_interface=enp24s0f0 compute02 network_interface=enp24s0f0 api_interface=eno1 storage_interface=enp26s0f0 tunnel_interface=enp24s0f0 compute03 network_interface=enp24s0f0 api_interface=eno1 storage_interface=enp26s0f0 tunnel_interface=enp24s0f0 [monitoring] controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 [storage] controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1 controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1

 

2.4 Kolla 설정 파일 세팅(2)

### Octavia 설치시 미리 함께 설치해놓기 $ vim /etc/kolla/globals.yml # enable_cinder_backend_lvm: "yes"  //cinder에 사용할 디스크가 lvm인 경우 yes # enable_cinder_backend_nfs: "yes"  // cinder에 사용할 디스크가 nfs인 경우 yes # enable_mariabackup: "yes"         //mariadb 백업 기능 사용 시 yes  # enable_neutron_provider_networks: "yes" # Octavia 설치 시 yes config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "ussuri" kolla_internal_vip_address: "172.21.0.100" kolla_external_vip_address: "192.168.0.X" enable_openstack_core: "yes" enable_glance: "{{ enable_openstack_core | bool }}" enable_haproxy: "yes" enable_keepalived: "{{ enable_haproxy | bool }}" enable_keystone: "{{ enable_openstack_core | bool }}" enable_mariadb: "yes" enable_memcached: "yes" enable_neutron: "{{ enable_openstack_core | bool }}" enable_nova: "{{ enable_openstack_core | bool }}" enable_rabbitmq: "{{ 'yes' if om_rpc_transport == 'rabbit' or om_notify_transport == 'rabbit' else 'no' }}" enable_chrony: "yes" enable_cinder: "yes" enable_cinder_backup: "yes" enable_cinder_backend_nfs: "yes" enable_fluentd: "yes" enable_heat: "{{ enable_openstack_core | bool }}" enable_horizon: "{{ enable_openstack_core | bool }}" enable_horizon_heat: "{{ enable_heat | bool }}" enable_horizon_octavia: "yes" enable_neutron_provider_networks: "yes" enable_nova_ssh: "yes" enable_octavia: "yes" enable_openvswitch: "no" enable_ovn: "{{ enable_neutron | bool and neutron_plugin_agent == 'ovn' }}" enable_placement: "{{ enable_nova | bool or enable_zun | bool }}" glance_backend_file: "yes" # Valid options are [ nfs, swift, ceph ] cinder_backup_driver: "nfs" cinder_backup_share: "NASIP:/cinder_backup" #cinder_backup_mount_options_nfs: "vers=3" nova_compute_virt_type: "kvm" nova_console: "novnc" ########################################################### Octavia Setting list - Octavia 추후 설치시 세팅 이후 재배포 ########################################################### octavia_loadbalancer_topology: "ACTIVE_STANDBY" octavia_amp_flavor_id: "100" octavia_amp_boot_network_list: "b19eb298-78bf-4e7a-a373-3ab7d3d46bad" octavia_amp_secgroup_list: "2934a87d-28b6-4960-9272-2adb9b4c3c43" ### Yes 목록 확인 $ grep -vE '^$|^#' /etc/kolla/globals.yml

 

2.5 Kolla 설정 파일 세팅(3)

$ kolla-genpwd $ vim /etc/kolla/passwords.yml  # DB 접속 시 편의를 위해 설치할 서비스들의 (서비스명)_database_password 를 “openstack” (혹은 사용할 패스워드) 로 변경.  Database_password: openstack //mariaDB 패스워드 변경 (원하는 패스워드) Keystone_admin_password: openstack //web 접속 시 admin의 password #아래는 Octavia 설치 시 변경 Octavia_ca_password: openstack Octavia_database_password: openstack Octavia_keystone_password: openstack

 

3. Openstack 설치

3.1 Openstack 배포

$ cd ~/inventory  $ kolla-ansible -i inventory/multinode bootstrap-servers  $ kolla-ansible -i inventory/multinode prechecks -vvv  $ kolla-ansible -i inventory/multinode deploy -vvv

 

4. Openstack 초기 설정

### Neutron MTU 값 설정 $ vim /etc/kolla/neutron-dhcp-agent/dnsmasq.conf dhcp-option-force=option:mtu,1400 또는 log-facility=/var/log/kolla/neutron/dnsmasq.log dhcp-option-force=26,1400 $ docker restart neutron_dhcp_agent ### Openstack CLI 설정 $ pip3 install python-openstackclient --ignore-installed PyYAML ### Warning 시 아래와 같이 해결 - 기능상 문제는 없음 $ pip3 install cryptography==3.3.2 ### admin-openrc.sh 생성 $ kolla-ansible post-deploy $ . /etc/kolla/admin-openrc.sh ### /etc/fstab 설정 $ vim /etc/fstab NASIP:/glance /var/lib/docker/volumes/glance/_data/images nfs defaults,_netdev 0 0  NASIP:/nova /var/lib/docker/volumes/nova_compute/_data/instances nfs defaults,_netdev 0 0

 

• 자동 스크립트 네트워크, 라우터 등 생성

### 자동 스크립트 파일 ### public network, private network, router, security group, flavor, cirros image 자동 생성 ### 한번 잘못 실행시 자원 다 지우고 cirros.img 도 삭제 이후 다시 가능 $ cp /usr/local/share/kolla-ansible/init-runonce /root/inventory/ $ cd /root/inventory # init-runonce 파일을 수정(public 네트워크 대역을 맞게 수정) $ vim init-runonce  …  # External network 생성을 위한 IP 대역, 범위, Gateway수정.  ENABLE_EXT_NET=${ENABLE_EXT_NET:-1} EXT_NET_CIDR='192.168.0.0/24' EXT_NET_RANGE='start=192.168.0.X,end=192.168.0.XX' EXT_NET_GATEWAY='192.168.0.X' …   # 위에서 정의된 내용으로 External network 생성 openstack network create --external --provider-physical-network physnet1 --provider-network-type flat public1 openstack subnet create --no-dhcp --allocation-pool ${EXT_NET_RANGE} --network public1 --subnet-range ${EXT_NET_CIDR} –gateway ${EXT_NET_GATEWAY} public1-subnet # Internal Network 생성 (demo-net이라는 이름으로 네트워크 생성됨. 이름 수정가능) openstack network create --provider-network-type vxlan demo-net openstack subnet create --subnet-range 10.0.0.0/24 --network private-net --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 demo-subnet # router 생성 및 위에서 생성한 subnet을 라우터에 연결(demo-router 라는 이름으로 라우터 생성됨.) openstack router create demo-router openstack router add subnet demo-router demo-subnet if [[ $ENABLE_EXT_NET -eq 1 ]]; then   openstack router set --external-gateway public1 demo-router # host의 공개키(id_rsa.pub)로 mykey라는 keypair를 생성하므로, host에서 별도의 키 지정이나 인증없이 mykey를 이용하여 생성한 vm으로 바로 ssh 접속 가능. openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey # 수정 후 init-runonce 스크립트 실행  $ . init-runonce

 

• 수동 스크립트 네트워크, 라우터 등 생성

# External네트워크 생성 $ openstack network create --external --provider-physical-network physnet1 --provider-network-type flat public1 # External 서브넷 생성  $ openstack subnet create --no-dhcp --allocation-pool start=192.168.0.X,end=192.168.0.XX --network public1 --subnet-range 192.168.0.X/24 –gateway 192.168.0.X public1-net # Internal 네트워크 생성 $ openstack network create private-net  # Internal 서브넷 생성  $ openstack subnet create --subnet-range 10.10.0.0/24 --network private-net --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 private-subnet # 라우터 생성 $ openstack router create router1 # 라우터에 서브넷 연결 $ openstack router add subnet router1 private-subnet # 라우터의 외부 게이트웨이를 public 네트워크로 설정.  $ openstack router set --external-gateway public1-net router1