반응형

* Openstack Octavia Install Guide(Xena - Ubuntu 20.04.2 LTS)

 

Octavia — kolla-ansible 14.1.0.dev62 documentation

Octavia Octavia provides load balancing as a service. This guide covers configuration of Octavia for the Amphora driver. See the Octavia documentation for full details. The installation guide is a useful reference. Enabling Octavia Enable the octavia servi

docs.openstack.org


1. Octavia 파일 생성

$ cp admin-openrc.sh octavia-openrc.sh
# Ansible managed

# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="}  /^OS_/ {print $1}' ); do unset $key ; done
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=service
export OS_TENANT_NAME=service
export OS_USERNAME=octavia
export OS_PASSWORD=openstack
export OS_AUTH_URL=http://172.16.0.110:35357/v3
export OS_INTERFACE=internal
export OS_ENDPOINT_TYPE=internalURL
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME=RegionOne
export OS_AUTH_PLUGIN=password

 

2. Amphora 이미지 생성

$ apt -y install debootstrap
$ git clone https://opendev.org/openstack/octavia -b stable/xena
$ pip3 install diskimage-builder
$ apt install debootstrap qemu-utils kpartx -y
$ cd /octavia/diskimage-create
$ ./diskimage-create.sh

$ openstack image create amphora-x64-haproxy.qcow2 --container-format bare --disk-format qcow2 --private --tag amphora --file amphora-x64-haproxy.qcow2

 

3. SSL 인증서 생성

### 다음 두가 빙법 설정
############################################################################################
* 자동 생성

$ vim /etc/kolla/globals.yml 

octavia_certs_country: KR
octavia_certs_state: Oregon
octavia_certs_organization: OpenStack
octavia_certs_organizational_unit: Octavia

$ kolla-ansible octavia-certificates

############################################################################################
* 수동 생성

$ mkdir -p /etc/kolla/config/octavia/certs
$ chmod -R 700 /etc/kolla/config/octavia/certs
$ cd octavia/bin/
$ cp openssl.cnf /etc/kolla/config/octavia/certs
$ cd /etc/kolla/config/octavia/certs
$ mkdir server_ca
$ mkdir client_ca 

$ cd server_ca/
$ mkdir certs crl newcerts private
$ chmod 700 private
$ touch index.txt
$ echo 1000 > serial

# 서버 CA key 생성
$ openssl genrsa -aes256 -out private/ca.key.pem 4096     -> password : openstack
$ chmod 400 private/ca.key.pem


# 서버 CA 인증서 생성
$ openssl req -config ../openssl.cnf -key private/ca.key.pem -new -x509 -days 7300 -sha256 -extensions v3_ca -out certs/ca.cert.pem
Country Name (2 letter code) [US]:US
State or Province Name [Oregon]:Oregon
Locality Name [Corvallis]:Corvallis
Organization Name [OpenStack]:openstack
Organizational Unit Name [Octavia]:octavia
Common Name [example.org]:openstack
Email Address []:


$ cd ../client_ca
$ mkdir certs crl csr newcerts private
$ chmod 700 private
$ touch index.txt
$ echo 1000 > serial

# 클라이언트 CA 키 생성
$ openssl genrsa -aes256 -out private/ca.key.pem 4096     -> password : openstack
$ chmod 400 private/ca.key.pem

# 클라이언트 CA 인증서 생성
$ openssl req -config ../openssl.cnf -key private/ca.key.pem -new -x509 -days 7300 -sha256 -extensions v3_ca -out certs/ca.cert.pem

# 클라이언트 CA 인증서 키 생성
$ openssl genrsa -aes256 -out private/client.key.pem 2048     -> password : openstack

$ openssl req -config ../openssl.cnf -new -sha256 -key private/client.key.pem -out csr/client.csr.pem

$ touch index.txt.attr 
# 클라이언트 인증서 요청에 서명
$ openssl ca -config ../openssl.cnf -extensions usr_cert -days 7300 -notext -md sha256 -in csr/client.csr.pem -out certs/client.cert.pem

# 연결된 클라이언트 인증서 및 키 파일 생성
$ openssl rsa -in private/client.key.pem -out private/client.cert-and-key.pem
$ cat certs/client.cert.pem >> private/client.cert-and-key.pem

##### 아래 칸 생략해도 됨
------------------------------------------------------------------------------------
$ cd ..
$ mkdir -p octavia/certs
$ chmod 700 octavia/certs/
$ cp server_ca/private/ca.key.pem octavia/certs/server_ca.key.pem
-> cp server_ca/private/ca.key.pem /etc/kolla/config/octavia/server_ca.key.pem
$ chmod 700 /etc/kolla/config/octavia/certs/server_ca.key.pem
------------------------------------------------------------------------------------
#####
$ cd /etc/kolla/config/octavia/certs/
$ cp client_ca/certs/ca.cert.pem /etc/kolla/config/octavia/client_ca.cert.pem
$ cp server_ca/certs/ca.cert.pem /etc/kolla/config/octavia/server_ca.cert.pem
$ cp server_ca/private/ca.key.pem /etc/kolla/config/octavia/server_ca.key.pem
$ cp client_ca/private/client.cert-and-key.pem  /etc/kolla/config/octavia/client.cert-and-key.pem 
$ cd ..
$ chmod 700 client.cert-and-key.pem
### $ chmod 700 octavia/certs/client.cert-and-key.pem -> 없음 제외

생성된 client_ca.cert.pem, client.cert-and-key.pem, server_ca.cert.pem, server_ca.key.pem 4개의 파일이 
/etc/kolla/config/octavia 디렉토리에 있으면 됨!

 

4. Octavia 배포

4.1 Octavia 설정

$ vim /etc/kolla/password.yml

# 원하는 password로 변경, ca_password는 인증서 생성시 password와 동일해야함. 

Octavia_ca_password: openstack
Octavia_database_password: openstack
Octavia_keystone_password: openstack

 

4.2 Octavia 자동 설치 설정

$ vim /etc/kolla/globals.yml
### 하단 내용 추가

enable_horizon_octavia: "yes"
enable_neutron_provider_networks: "yes"
enable_octavia_driver_agent: "{{ enable_octavia | bool and neutron_plugin_agent == 'ovn' }}"
enable_octavia: "yes"

octavia_auto_configure: yes
octavia_amp_flavor:
  name: "m1.amphora"
  is_public: yes
  flavorid: 100
  vcpus: 2
  ram: 2048
  disk: 10
octavia_amp_security_groups:
    mgmt-sec-grp:
      name: "octavia-sec"
      enabled: yes
      rules:
        - protocol: icmp
        - protocol: tcp
          src_port: 22
          dst_port: 22
        - protocol: tcp
          src_port: "9443"
          dst_port: "9443"
octavia_amp_network:
  name: octavia-net
  shared: true
  #provider_network_type: geneve  ### 자동으로 geneve로 설정됨
  subnet:
      name: octavia-sub
      cidr: "{{ octavia_amp_network_cidr }}"
      #allocation_pool_start: "20.0.0.11"  ### 범위 설정 해도 되고 안해도됨
      #allocation_pool_end: "20.0.0.100"
      gateway_ip: "20.0.0.1"
      no_gateway_ip: no
      enable_dhcp: yes
octavia_amp_network_cidr: 20.0.0.0/24

octavia_amp_image_tag: "amphora"
octavia_loadbalancer_topology: "SINGLE"

octavia_certs_country: KR
octavia_certs_state: Oregon
octavia_certs_organization: OpenStack
octavia_certs_organizational_unit: Octavia

 

4.3 Network 포트 설정

$ sudo docker exec -it openvswitch_vswitchd bash
$ pip3 install python-neutronclient
$ vi octavia-openrc.sh


$ source octavia-openrc.sh

######### 수동설치시 다음 내용 설정하지만 자동 설치시는 하지 않는다.#####################################
$ OCTAVIA_MGMT_SUBNET=30.0.0.0/24
$ OCTAVIA_MGMT_SUBNET_START=30.0.0.101
$ OCTAVIA_MGMT_SUBNET_END=30.0.0.200
$ OCTAVIA_AMP_NETWORK_ID=$(neutron net-create lb-mgmt-net | awk '/ id / {print $4}')
$ neutron subnet-create --name lb-mgmt-subnet --allocation-pool start=$OCTAVIA_MGMT_SUBNET_START,end=$OCTAVIA_MGMT_SUBNET_END lb-mgmt-net $OCTAVIA_MGMT_SUBNET
############################################################################################

######################### 멀티 Controller일 경우 모든 노드에 설정################################
(openvswitch-vswitchd)[root@lsmopensteack /] $ neutron port-create --name octavia-hm-port --binding:host_id=$HOSTNAME octavia-net
(openvswitch-vswitchd)[root@lsmopensteack /] $ MGMT_PORT_ID=$(neutron port-show octavia-hm-port | awk '/ id / {print $4}')
(openvswitch-vswitchd)[root@lsmopensteack /] $ MGMT_PORT_MAC=$(neutron port-show octavia-hm-port | awk '/ mac_address / {print $4}')

(openvswitch-vswitchd)[root@lsmopensteack /] $ sudo ovs-vsctl -- --may-exist add-port br-int octavia-hm0 -- set Interface octavia-hm0 type=internal -- set Interface octavia-hm0 external-ids:iface-status=active -- set Interface octavia-hm0 external-ids:attached-mac=$MGMT_PORT_MAC -- set Interface octavia-hm0 external-ids:iface-id=$MGMT_PORT_ID
(openvswitch-vswitchd)[root@lsmopensteack /] $ ip link set dev octavia-hm0 address $MGMT_PORT_MAC
(openvswitch-vswitchd)[root@lsmopensteack /] $ exit

root@lsmopensteack:~$ HM_IP=$(openstack port show octavia-hm-port | awk '/ fixed_ips / {print $4}' | cut -d "'" -f 2)
root@lsmopensteack:~$ echo $HM_IP
20.0.0.X

root@lsmopensteack:~$ ifconfig octavia-hm0 20.0.0.X/24
##################################################################################################################


### 하단 내용은 LB 생성시 운영 상태를 온라인으로 변경하는 세팅이며 미설정시 운영 상태는 오프라인으로 표기.
### 멀티노드인 경우 각 노드별 설정 필요
$ vim /etc/kolla/config/octavia.conf

[health_manager]
bind_ip = {Controller External IP}
controller_ip_port_list = {Controller External IP}:5555

 

4.4 Reconfigure Octavia

$ kolla-ansible -i inventory/all-in-one deploy -t octavia
$ kolla-ansible -i inventory/multinode deploy -t octavia

$ kolla-ansible -i inventory/all-in-one reconfigure -t octavia
$ kolla-ansible -i inventory/multinode reconfigure -t octavia

 

4.5 추가설정

1. deploy 이후 생성되는 octavia-net를 기본 라우터에 추가한다.
2. octavia-net에서 생성된 octavia-hm-port가 active인지 확인한다.
3. 보안그룹은 확인이후 ICMP, TCP 모두 허용 설정
4. 위 가이드는 global.yml 파일의 설정을 바탕으로 octavia 자동 설치 설정 가이드이다.
# 수동 설치는 https://githubhot.com/repo/prastamaha/openstack-octavia 참고
5. https://www.notion.so/miners1205/Openstack-Install-Guide-Xena-Ubuntu-20-04-Netplan-ddca795edd264970b5c4d89648c32b88 가이드와 함께 확인

 

반응형
반응형

Ussuri Version - MariaDB

  • mysql Ver 15.1 Distrib 10.3.32-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

Docker version

  • Docker version 20.10.11, build dea9396

 

1. 물리 환경 및 네트워크 구성

1.1 물리 환경

  • 4 Network Interface(External, MGMT, DATA, STORAGE)
  • 8GB Main Memory
  • 40GB Disk Space

Openstack kolla-ansible Version 10.4.0(Ussuri)

OS Ubuntu 18.04.5
NIC External, MGMT, DATA, STORAGE

 

1.2 네트워크 설정

1) root 계정 활성화

# root 계정 암호 설정 및 로그인
$ sudo passwd root

 

2) 방화벽 중지

$ ufw disable
$ ufw status
Status: inactive

 

3) ipv6 중지

$ vim /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

 

4) SSH 키 분배

$ ssh-keygen -t rsa
$ ssh-copy-id root@hosts

 

5) OVS 설치 및 설정

$ apt install openvswitch-switch

$ systemctl status openvswitch-switch

$ vim /lib/systemd/system/ovsdb-server.service

### Unit 기존 주석 및 신규 Unit 추가
### 이유 : 재부팅시 OVS 브릿지 보다 일반 네트워크가 먼저 올라와 브릿지 인터페이스가 시작되지 않음

#[Unit]
#Description=Open vSwitch Database Unit
#After=syslog.target network-pre.target
#Before=network.target network.service
#ReloadPropagatedFrom=openvswitch-switch.service
#PartOf=openvswitch-switch.service

[Unit]
Description=Open vSwitch Database Unit
After=syslog.target network-pre.target dpkg.service local-fs.target
Before=network.target network.service
PartOf=openvswitch-switch.service
DefaultDependencies=no

[Service]
LimitNOFILE=1048576
Type=forking
Restart=on-failure
EnvironmentFile=-/etc/default/openvswitch-switch
ExecStart=/usr/share/openvswitch/scripts/ovs-ctl \\
--no-ovs-vswitchd --no-monitor --system-id=random \\
start $OVS_CTL_OPTS
ExecStop=/usr/share/openvswitch/scripts/ovs-ctl --no-ovs-vswitchd stop
ExecReload=/usr/share/openvswitch/scripts/ovs-ctl --no-ovs-vswitchd \\
--no-monitor restart $OVS_CTL_OPTS
RuntimeDirectory=openvswitch
RuntimeDirectoryMode=0755

 

6) NAS 연결 패키지 설치 및 시간 설정

$ apt-get install nfs-kernel-server

$ mkdir -p /etc/kolla/config/nfs_shares

$ vim /etc/kolla/config/nfs_shares

NASIP:/NFS/os/cinder

$ timedatectl set-timezone 'Asia/Seoul'

 

7) networking 서비스 사용을 위한 패키지 설치

$ apt-get install ifupdown
$ vim /etc/network/interfaces 

### Interface 확인 및 브릿지 설정
auto lo
iface lo inet loopback

#####################
External 네트워크 세팅 
#####################
auto br-ex
allow-ovs br-ex
iface br-ex inet static
address 192.168.0.X ~ 192.168.0.XX
netmask 255.255.255.0
gateway 192.168.0.X
dns-nameservers 8.8.8.8
ovs_type OVSBridge
ovs_ports enp26s0f1

auto enp26s0f1
allow-br-ex enp26s0f1
iface enp26s0f1 inet manual
ovs_bridge br-ex
ovs_type OVSPort

#####################
DATA 네트워크 세팅 - 사설망
#####################
auto br-int
allow-ovs br-int
iface br-int inet static
address 172.22.0.1 ~ 172.22.0.12
netmask 255.255.255.0
ovs_type OVSBridge
ovs_ports br-int

auto enp24s0f0
allow-br-int enp24s0f0
iface enp24s0f0 inet static
address 172.22.0.101 ~ 172.22.0.112
netmask 255.255.255.0
ovs_bridge br-int
ovs_type OVSPort

#####################
MGMT 네트워크 세팅 - 사설망
#####################
auto eno1
iface eno1 inet static
address 172.21.0.1 ~ 172.21.0.12
netmask 255.255.255.0

#####################
STORAGE 네트워크 세팅 - 사설망
#####################
auto enp26s0f0
iface enp26s0f0 inet static
address 172.18.0.1 ~ 172.18.0.12

 

8) modprobe 설정

$ modprobe ip_vs

$ modprobe ip6_tables

$ vi /etc/modules-load.d/ip_vs.conf
ip_vs

$ vi /etc/modules-load.d/ip6_tables.conf
ip6_tables

 

9) 파이썬 버전 설정

### 파이썬 3버전 설정
$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1

$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2

$ update-alternatives --config python

 

2. Kolla 설치 준비 & 설치

2.1 Kolla 의존성 설치

$ apt-get install python3-dev libffi-dev gcc libssl-dev

$ pip3 install ‘ansible<2.10’

$ pip3 install ansible==2.9.6

$ apt install ansible

$ vim /etc/ansible/ansible.cnf

[defaults]
forks          = 100
host_key_checking = False

[ssh_connection]
pipelining = True

 

2.2 Kolla 설치

$ pip install 'kolla-ansible==10.4.0' --ignore-installed PyYAML

$ cp -r /usr/local/share/kolla-ansible/etc_examples/kolla /etc/

$ cp -r /usr/local/share/kolla-ansible/ansible/inventory/ /root/

 

2.3 Kolla 설정 파일 세팅(1)

<컨트롤노드 호스트명> network_interface=<내부물리NIC이름> neutron_external_interface=<외부물리NIC이름> kolla_external_vip_interface=<외부물리NIC이름>
<컴퓨트노드 호스트명> network_interface=<내부물리NIC이름> api_interface=<관리(없는 경우 내부 물리)NIC이름> storage_interface=<내부물리NIC이름> tunnel_interface=<내부물리NIC이름>

$ cd /root/inventory
$ vim /root/inventory/multinode

# 그 외 설치 패키지에 따라 세팅 변경

[control]
controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1

[network]
controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1

[compute]
compute01 network_interface=enp24s0f0 api_interface=eno1 storage_interface=enp26s0f0 tunnel_interface=enp24s0f0
compute02 network_interface=enp24s0f0 api_interface=eno1 storage_interface=enp26s0f0 tunnel_interface=enp24s0f0
compute03 network_interface=enp24s0f0 api_interface=eno1 storage_interface=enp26s0f0 tunnel_interface=enp24s0f0

[monitoring]
controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1

[storage]
controller01 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller02 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1
controller03 network_interface=enp24s0f0 api_interface=eno1 neutron_external_interface=enp26s0f1 kolla_external_vip_interface=enp26s0f1

 

2.4 Kolla 설정 파일 세팅(2)

### Octavia 설치시 미리 함께 설치해놓기

$ vim /etc/kolla/globals.yml

# enable_cinder_backend_lvm: "yes"  //cinder에 사용할 디스크가 lvm인 경우 yes
# enable_cinder_backend_nfs: "yes"  // cinder에 사용할 디스크가 nfs인 경우 yes
# enable_mariabackup: "yes"         //mariadb 백업 기능 사용 시 yes 
# enable_neutron_provider_networks: "yes" # Octavia 설치 시 yes

config_strategy: "COPY_ALWAYS"
kolla_base_distro: "ubuntu"
kolla_install_type: "source"
openstack_release: "ussuri"
kolla_internal_vip_address: "172.21.0.100"
kolla_external_vip_address: "192.168.0.X"
enable_openstack_core: "yes"
enable_glance: "{{ enable_openstack_core | bool }}"
enable_haproxy: "yes"
enable_keepalived: "{{ enable_haproxy | bool }}"
enable_keystone: "{{ enable_openstack_core | bool }}"
enable_mariadb: "yes"
enable_memcached: "yes"
enable_neutron: "{{ enable_openstack_core | bool }}"
enable_nova: "{{ enable_openstack_core | bool }}"
enable_rabbitmq: "{{ 'yes' if om_rpc_transport == 'rabbit' or om_notify_transport == 'rabbit' else 'no' }}"
enable_chrony: "yes"
enable_cinder: "yes"
enable_cinder_backup: "yes"
enable_cinder_backend_nfs: "yes"
enable_fluentd: "yes"
enable_heat: "{{ enable_openstack_core | bool }}"
enable_horizon: "{{ enable_openstack_core | bool }}"
enable_horizon_heat: "{{ enable_heat | bool }}"
enable_horizon_octavia: "yes"
enable_neutron_provider_networks: "yes"
enable_nova_ssh: "yes"
enable_octavia: "yes"
enable_openvswitch: "no"
enable_ovn: "{{ enable_neutron | bool and neutron_plugin_agent == 'ovn' }}"
enable_placement: "{{ enable_nova | bool or enable_zun | bool }}"
glance_backend_file: "yes"

# Valid options are [ nfs, swift, ceph ]
cinder_backup_driver: "nfs"
cinder_backup_share: "NASIP:/cinder_backup"
#cinder_backup_mount_options_nfs: "vers=3"

nova_compute_virt_type: "kvm"
nova_console: "novnc"
###########################################################
Octavia Setting list - Octavia 추후 설치시 세팅 이후 재배포
###########################################################
octavia_loadbalancer_topology: "ACTIVE_STANDBY"
octavia_amp_flavor_id: "100"
octavia_amp_boot_network_list: "b19eb298-78bf-4e7a-a373-3ab7d3d46bad"
octavia_amp_secgroup_list: "2934a87d-28b6-4960-9272-2adb9b4c3c43"

### Yes 목록 확인
$ grep -vE '^$|^#' /etc/kolla/globals.yml

 

2.5 Kolla 설정 파일 세팅(3)

$ kolla-genpwd
$ vim /etc/kolla/passwords.yml 

# DB 접속 시 편의를 위해 설치할 서비스들의 (서비스명)_database_password 를 “openstack” (혹은 사용할 패스워드) 로 변경. 

Database_password: openstack //mariaDB 패스워드 변경 (원하는 패스워드)
Keystone_admin_password: openstack //web 접속 시 admin의 password

#아래는 Octavia 설치 시 변경
Octavia_ca_password: openstack
Octavia_database_password: openstack
Octavia_keystone_password: openstack

 

3. Openstack 설치

3.1 Openstack 배포

$ cd ~/inventory 
$ kolla-ansible -i inventory/multinode bootstrap-servers 
$ kolla-ansible -i inventory/multinode prechecks -vvv 
$ kolla-ansible -i inventory/multinode deploy -vvv

 

4. Openstack 초기 설정

### Neutron MTU 값 설정

$ vim /etc/kolla/neutron-dhcp-agent/dnsmasq.conf
dhcp-option-force=option:mtu,1400

또는
log-facility=/var/log/kolla/neutron/dnsmasq.log
dhcp-option-force=26,1400

$ docker restart neutron_dhcp_agent

### Openstack CLI 설정
$ pip3 install python-openstackclient --ignore-installed PyYAML

### Warning 시 아래와 같이 해결 - 기능상 문제는 없음
$ pip3 install cryptography==3.3.2

### admin-openrc.sh 생성
$ kolla-ansible post-deploy
$ . /etc/kolla/admin-openrc.sh

### /etc/fstab 설정
$ vim /etc/fstab
NASIP:/glance /var/lib/docker/volumes/glance/_data/images nfs defaults,_netdev 0 0 
NASIP:/nova /var/lib/docker/volumes/nova_compute/_data/instances nfs defaults,_netdev 0 0

 

  • 자동 스크립트 네트워크, 라우터 등 생성
### 자동 스크립트 파일
### public network, private network, router, security group, flavor, cirros image 자동 생성
### 한번 잘못 실행시 자원 다 지우고 cirros.img 도 삭제 이후 다시 가능

$ cp /usr/local/share/kolla-ansible/init-runonce /root/inventory/
$ cd /root/inventory

# init-runonce 파일을 수정(public 네트워크 대역을 맞게 수정)
$ vim init-runonce 
… 
# External network 생성을 위한 IP 대역, 범위, Gateway수정. 
ENABLE_EXT_NET=${ENABLE_EXT_NET:-1}
EXT_NET_CIDR='192.168.0.0/24'
EXT_NET_RANGE='start=192.168.0.X,end=192.168.0.XX'
EXT_NET_GATEWAY='192.168.0.X'

  # 위에서 정의된 내용으로 External network 생성
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat public1
openstack subnet create --no-dhcp --allocation-pool ${EXT_NET_RANGE} --network public1 --subnet-range ${EXT_NET_CIDR} –gateway ${EXT_NET_GATEWAY} public1-subnet

# Internal Network 생성 (demo-net이라는 이름으로 네트워크 생성됨. 이름 수정가능)
openstack network create --provider-network-type vxlan demo-net
openstack subnet create --subnet-range 10.0.0.0/24 --network private-net --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 demo-subnet

# router 생성 및 위에서 생성한 subnet을 라우터에 연결(demo-router 라는 이름으로 라우터 생성됨.)
openstack router create demo-router
openstack router add subnet demo-router demo-subnet
if [[ $ENABLE_EXT_NET -eq 1 ]]; then
  openstack router set --external-gateway public1 demo-router
# host의 공개키(id_rsa.pub)로 mykey라는 keypair를 생성하므로, host에서 별도의 키 지정이나 인증없이 mykey를 이용하여 생성한 vm으로 바로 ssh 접속 가능.
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey

# 수정 후 init-runonce 스크립트 실행 
$ . init-runonce

 

  • 수동 스크립트 네트워크, 라우터 등 생성
# External네트워크 생성
$ openstack network create --external --provider-physical-network physnet1 --provider-network-type flat public1

# External 서브넷 생성 
$ openstack subnet create --no-dhcp --allocation-pool start=192.168.0.X,end=192.168.0.XX --network public1 --subnet-range 192.168.0.X/24 –gateway 192.168.0.X public1-net

# Internal 네트워크 생성
$ openstack network create private-net 

# Internal 서브넷 생성 
$ openstack subnet create --subnet-range 10.10.0.0/24 --network private-net --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 private-subnet

# 라우터 생성
$ openstack router create router1

# 라우터에 서브넷 연결
$ openstack router add subnet router1 private-subnet

# 라우터의 외부 게이트웨이를 public 네트워크로 설정. 
$ openstack router set --external-gateway public1-net router1

 

반응형
반응형

Openstack Install Guide(Victoria - CentOS)

  • CentOS 8.2 환경 기준 설치

 

1. OS 환경

1.1 CentOS 8.2

1.2 All-in-One 기준 구성

1.3 기본적으로 네트워크 외부 통신 연결 가능상태

1.4 네트워크 포토 최소 2~4개 필요

2. Openstack OS 기본 설정

  • 모든 서버에 같은 설정
  • 네트워크 서비스 기본 설정 완료 상태

 

2.1 기본 툴 설치

$ yum install vim net-tools -y          ### 기본 툴
$ yum install nfs-utils -y                 ### NAS 스토리지 연결시 필요

 

2.2 네트워크 설정

  • IPv6 중지 및 설정 확인
$ vim /etc/sysctl.conf 
net.ipv6.conf.all.disable_ipv6 = 1 
net.ipv6.conf.default.disable_ipv6 = 1 
net.ipv6.conf.lo.disable_ipv6 = 1

$ sysctl -p  
  • 방화벽 중지
$ systemctl stop firewalld
$ systemctl disable firewalld

2.3 SSH KEY 설정

  • 모든 서버 동시 수행
$ ssh-keygen -t rsa
$ ssh-copy-id root@{모든 서버}

 

2.4 SELINUX 비활성화

$ vim /etc/selinux/config

  SELINUX=disabled

$ reboot

 

2.5 Openvswitch 설치 및 패키지 설치

$ dnf install epel-release -y

$ dnf install python3-devel libffi-devel gcc openssl-devel python3-libselinux -y

$ dnf install python3-pip -y

$ pip3 install -U pip

$ dnf install ansible        ### (==2.9.18)  
### 2021.03.30 2.9.18 ansible version 설치
### 2021.07.02 2.9.23 ansible version 설치

$ pip3 install 'ansible==2.9.18'

### $ yum install -y centos-release-openstack-victoria  ### 참고
-> OVS 설치 레포

$ yum install -y openvswitch

$ modprobe ip_vs

$ modprobe ip6_tables

$ modprobe openvswitch

$ vi /etc/modules-load.d/ip_vs.conf 
   ip_vs 

$ vi /etc/modules-load.d/ip6_tables.conf 
   ip6_tables

$ systemctl enable openvswitch

$ systemctl start openvswitch

$ yum install network-scripts

### centos7 과 동일 하게 사용
$ service network start  

3. Openstack 설치 구성

3.1 Openstack 설치 패키지 구성

 

### kolla-ansible 구성 설치

### Openstack Victoria 버전 
$ pip3 install kolla-ansible==11.0.0   


### ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

### 에러 발생시 대처
$ pip3 install --ignore-installed PyYAML

$ mkdir -p /etc/kolla

$ cp -r /usr/local/share/kolla-ansible/etc_examples/kolla/* /etc/kolla

$ cp -r /usr/local/share/kolla-ansible/ansible/inventory/ /root/


$ vim /etc/ansible/ansible.cfg
 [defaults]
 host_key_checking=False
 pipelining=True
 forks=100

3.2 All-in-One 노드 설정

$ cd /root/inventory 

### localhost -> {hostname} 으로 변경
$ sed -i s/localhost/호스트네임/g all-in-one

$ vim /root/inventory/all-in-one
[control]
all01       ansible_connection=local

[network]
all01       ansible_connection=local

[compute]
all01       ansible_connection=local

[storage]
all01       ansible_connection=local

[monitoring]
all01       ansible_connection=local

[deployment]
all01       ansible_connection=local

 

3.3 Kolla globals.yml 파일 설정

**### All-in-One 경우**
$ vim /etc/kolla/globals.yml

kolla_base_distro: "centos"           ### Docker 운영체제 선택  
kolla_install_type: "source"          ### Source, binary 설치 선택 
openstack_release: "victoria"         ### openstack 선택

kolla_internal_vip_address: "10.10.10.10"      ### 관리 network VIP 입력, 관리 network가 따로 없는 경우에는 Internal network VIP  
kolla_external_vip_address: "192.168.0.XXX"  ### External Network VIP 
network_interface: "enp1s0f1"                  ### 내부 물리 NIC (internal network(data))
kolla_external_vip_interface: "enp1s0f0"       ### 외부 물리 NIC
api_interface: "enp1s0f2"                      ### 관리 NIC (관리 네트워크 없는 경우 =network_interface) 
octavia_network_interface: "enp1s0f0"          ### 외부 물리 NIC
neutron_external_interface: "enp1s0f0"         ### 외부 물리 NIC

# 사용할 서비스를 enable(주석 해제, yes)하면됨. 
enable_cinder: "yes"
enable_cinder_backup: "yes"
# enable_cinder_backend_lvm: "yes"  # cinder에 사용할 디스크가 lvm인 경우 yes
# enable_cinder_backend_nfs: "yes"  # cinder에 사용할 디스크가 nfs인 경우 yes
# enable_horizon_neutron_lbaas: "no"# Octavia 설치 시 no 
# enable_horizon_octavia: "yes"     # Octavia 설치 시 yes
# enable_mariabackup: "yes"         # mariadb 백업 기능 사용 시 yes 
# enable_neutron_lbaas: "yes"       # Octavia 설치 시 yes
enable_openstack_core: "yes"
enable_openvswitch: "no" 
glance_enable_rolling_upgrade: "no"
nova_compute_virt_type: "kvm"       # vmware 등 가상환경에 설치 시 qemu로 변경

# database_port: 33306              # mariadb port를 변경하고 싶은 경우    

################################
# Cinder - Block Storage Options
################################

# Cinder backup 디스크가 NFS 인 경우 수정
# Valid options are [ nfs, swift, ceph ]
cinder_backup_driver: "nfs"
cinder_backup_share: "NASIP:/nfs/cinder_backup"
cinder_backup_mount_options_nfs: "vers=3"

# 입력한 정보 확인
$ grep -vE '^$|^#' /etc/kolla/globals.yml

### Multinode 경우
$ vim /etc/kolla/globals.yml

###############
# Kolla options
############### 
kolla_base_distro: "centos"  //docker에 올릴 운영체제
kolla_install_type: "source":Lq
openstack_release: "stein"   //openstack 버전 이름

kolla_internal_vip_address: "172.16.0.250" # 관리 network VIP /관리 network가 따로 없는 경우에는 Internal network VIP를 입력
kolla_external_vip_address: "192.168.0.XXX" # External Network VIP

# Multimode 파일에 NIC 인터페이스를 정의하였으므로 설정할 필요 X


# 사용할 서비스를 enable(주석 해제, yes)하면됨. 
enable_cinder: "yes"
enable_cinder_backup: "yes"
# enable_cinder_backend_lvm: "yes"  //cinder에 사용할 디스크가 lvm인 경우 yes
# enable_cinder_backend_nfs: "yes"  // cinder에 사용할 디스크가 nfs인 경우 yes
# enable_horizon_neutron_lbaas: "no"//Octavia 설치 시 no 
# enable_horizon_octavia: "yes"     //Octavia 설치 시 yes
# enable_mariabackup: "yes"         //mariadb 백업 기능 사용 시 yes 
# enable_neutron_lbaas: "yes"       //Octavia 설치 시 yes
enable_openstack_core: "yes"
enable_openvswitch: "no" 
glance_enable_rolling_upgrade: "no"
nova_compute_virt_type: "kvm"       //vmware 등 가상환경에 설치 시 qemu로 변경


################################
# Cinder - Block Storage Options
################################

# Cinder backup 디스크가 NFS 인 경우 수정
# Valid options are [ nfs, swift, ceph ]
cinder_backup_driver: "nfs"
cinder_backup_share: "NASIP:/nfs/cinder_backup"
cinder_backup_mount_options_nfs: "vers=3"

3.4 Kolla Password 설정

$ kolla-genpwd


### Openstack 서비스들의 암호를 설정하는것
$ vim /etc/kolla/passwords.yml
Database_password: openstack ### MariaDB 접속 암호 설정
Keystone_admin_password: openstack ### Horizon 접속 암호 설정


### 하단 Octavia 설치시 암호 설정
Octavia_ca_password: openstack
Octavia_database_password: openstack
Octavia_keystone_password: openstack

### 추가 서비스들에 대하여 변경하여 사용 필요.

3.5 Cinder Volume 구성

$ pvcreate /dev/sdb1

### cinder-volumes 명칭으로 구성해야한다. 명칭 변경시 하단 변경 필요
$ vgcreate cinder-volumes /dev/sdb1

3.6 Openstack 설치

$ cd ~/inventory 

### All-in-One인 경우
$ kolla-ansible -i all-in-one bootstrap-servers 
$ kolla-ansible -i all-in-one prechecks -vvv 
$ kolla-ansible -i all-in-one deploy -vvv

###  MultiNode인 경우
$ kolla-ansible -i multinode bootstrap-servers
$ kolla-ansible -i multinode prechekcs -vvv
$ kolla-ansible -i multinode deploy -vvv
$ mount /dev/sdb /var/lib/docker/volumes/nova_compute/_data/instances

### 내부 디스크 사용시 instances 권한 42436으로 변경 필요
$ chown 42436:42436 /var/lib/docker/volumes/nova_compute/_data/instances

 

  • Openstack CLI 설치
$ kolla-ansible post-deploy
$ pip install python-openstackclient

$ openstack --version
$ source /etc/kolla/admin-openrc.sh

 

  • Openstack Configuration
Openstack Router Failover

# vim /etc/kolla/neutron-l3-agent/neutron.conf
[Default]
allow_automatic_l3agent_failover = true

Openstack dnsmasq MTU 값 조정
# vim /etc/kolla/neutron-dhcp-agent/dnsmasq.conf
log-facility=/var/log/kolla/neutron/dnsmasq.log
dhcp-option-force=option:mtu,1400

# docker restart neutron_dhcp_agent

 

반응형
반응형

1. Openstack Port List 정리

가끔 필요에 의해서 사용하기에 Openstack 공홈에서 해당 내용을 가져왔다.

o Openstack ort list 목록

Openstack Service Default Port
Application Catalog (murano) 8082
Backup Service (Freezer) 9090
Big Data Processing Framework (sahara) 8386
Block Storage (cinder) 8776
Clustering (senlin) 8777
Compute (nova) endpoints 8774
Compute ports for access to virtual machine consoles 5900-5999
Compute VNC proxy for browsers (openstack-nova-novncproxy) 6080
Compute VNC proxy for traditional VNC clients (openstack-nova-xvpvncproxy) 6081
Container Infrastructure Management (Magnum) 9511
Container Service (Zun) 9517
Data processing service (sahara) endpoint 8386
Database service (Trove) 8779
DNS service (Designate) 9001
High Availability Service (Masakari) 15868
Identity service (keystone) endpoint 5000
Image service (glance) API 9292
Key Manager service (Barbican) 9311
Loadbalancer service (Octavia) 9876
Networking (neutron) 9696
NFV Orchestration service (tacker) 9890
Object Storage (swift) 6000, 6001, 6002
Orchestration (heat) endpoint 8004
Orchestration AWS CloudFormation-compatible API (openstack-heat-api-cfn) 8000
Orchestration AWS CloudWatch-compatible API (openstack-heat-api-cloudwatch) 8778
Placement API (placement) 8003
Proxy port for HTML5 console used by Compute service 6082
Rating service (Cloudkitty) 8889
Registration service (Adjutant) 5050
Resource Reservation service (Blazar) 1234
Root Cause Analysis service (Vitrage) 8999
Shared File Systems service (Manila) 8786
Telemetry alarming service (Aodh) 8042
Telemetry event service (Panko) 8977
Workflow service (Mistral) 8989

 

ServiceDefault portUsed by

Service Default Port Used by
HTTP 80 OpenStack dashboard (Horizon) when it is not configured to use secure access.
HTTP alternate 8080 OpenStack Object Storage (swift) service.
HTTPS 443 Any OpenStack service that is enabled for SSL, especially secure-access dashboard.
rsync 873 OpenStack Object Storage. Required.
iSCSI target 3260 OpenStack Block Storage. Required.
MySQL database service 3306 Most OpenStack components.
Message Broker (AMQP traffic) 5672 OpenStack Block Storage, Networking, Orchestration, and Compute.

 

 

* 참고

https://docs.openstack.org/install-guide/firewalls-default-ports.html

 

Firewalls and default ports — Installation Guide documentation

Firewalls and default ports On some deployments, such as ones where restrictive firewalls are in place, you might need to manually configure a firewall to permit OpenStack service traffic. To manually configure a firewall, you must permit traffic through t

docs.openstack.org

 

반응형
반응형

* Delete Load Balance

LB 생성시에는 정상적이지만 삭제시는 CLI로 삭제해야하며 모니터, 풀, 멤버, 리스너 모두 삭제를 해주어야 LB가 정상적으로 삭제가 된다.

* healthmonitor -> member -> pool -> listener -> LB 순으로 삭제

Delete healthmonitor

(neutron) lbaas-healthmonitor-list
+--------------------------------------------------------+---------+----------------------------------------------------+--------+------------------------+
| id                                                                 | name | tenant_id                                                  | type   | admin_state_up |
+--------------------------------------------------------+---------+----------------------------------------------------+--------+------------------------+
| 9201ca04-1e9a-47eb-b0c1-161853fdc4b1 |           | e4cb70dd38b44665bd20dce52e5c8c51 | PING | True                    |
+--------------------------------------------------------+---------+----------------------------------------------------+--------+------------------------+

(neutron) lbaas-healthmonitor-delete 9201ca04-1e9a-47eb-b0c1-161853fdc4b1
Deleted lbaas_healthmonitor(s): 9201ca04-1e9a-47eb-b0c1-161853fdc4b1
Delet Member & Pool
### 삭제하려는 pool ID 확인

(neutron) lbaas-pool-list
+--------------------------------------------------------+---------+----------------------------------------------------+-----------------------+------------+---------------------+
| id                                                                | name   | tenant_id                                               | lb_algorithm         | protocol | admin_state_up |
+--------------------------------------------------------+---------+----------------------------------------------------+-----------------------+------------+---------------------+
| c65fbc5a-2ff1-453d-8e52-9278288cf232 | Pool 1 | e4cb70dd38b44665bd20dce52e5c8c51 | ROUND_ROBIN  | HTTP     | True                  |
+--------------------------------------------------------+---------+----------------------------------------------------+-----------------------+------------+---------------------+

### pool의 멤버 확인

(neutron) lbaas-member-list c65fbc5a-2ff1-453d-8e52-9278288cf232
+--------------------------------------+------+----------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
| id                                                               | name | tenant_id                        | address   | protocol_port | weight | subnet_id                            | admin_state_up |
+--------------------------------------+------+----------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
| f0c66904-9816-49af-ab42-d87c73914407 |      | e4cb70dd38b44665bd20dce52e5c8c51 | 10.0.0.11 |            80 |      1 | da6d1aef-e1fa-4cf4-b475-ef80bcb5466f | True           |
| 1057cb5d-e9c8-4c0e-ba36-8797fa54d1db |      | e4cb70dd38b44665bd20dce52e5c8c51 | 10.0.0.18 |            80 |      1 | da6d1aef-e1fa-4cf4-b475-ef80bcb5466f | True           |
+--------------------------------------+------+----------------------------------+-----------+---------------+--------+--------------------------------------+----------------+

### 해당 멤버 삭제
### 삭제시 lbaas-member-delete 'member id' 'pool id'형식

(neutron) lbaas-member-delete f0c66904-9816-49af-ab42-d87c73914407 c65fbc5a-2ff1-453d-8e52-9278288cf232
Deleted lbaas_member(s): f0c66904-9816-49af-ab42-d87c73914407

(neutron) lbaas-member-delete 1057cb5d-e9c8-4c0e-ba36-8797fa54d1db c65fbc5a-2ff1-453d-8e52-9278288cf232
Deleted lbaas_member(s): 1057cb5d-e9c8-4c0e-ba36-8797fa54d1db

### 해당 풀 삭제
(neutron) lbaas-pool-delete c65fbc5a-2ff1-453d-8e52-9278288cf232
Deleted lbaas_pool(s): c65fbc5a-2ff1-453d-8e52-9278288cf232

 

### listener 삭제

(neutron) lbaas-listener-list
+--------------------------------------+-----------------+------------+----------------------------------+----------+---------------+----------------+
| id                                   | default_pool_id | name       | tenant_id                        | protocol | protocol_port | admin_state_up |
+--------------------------------------+-----------------+------------+----------------------------------+----------+---------------+----------------+
| 19496e80-1af8-44d3-8028-7af2ade87a19 |                 | Listener 1 | e4cb70dd38b44665bd20dce52e5c8c51 | HTTP     |            80 | True           |
+--------------------------------------+-----------------+------------+----------------------------------+----------+---------------+----------------+
(neutron)
(neutron) lbaas-listener-delete 19496e80-1af8-44d3-8028-7af2ade87a19
Deleted listener(s): 19496e80-1af8-44d3-8028-7af2ade87a19

 

### Load Balance 삭제

(neutron) lbaas-loadbalancer-list
+--------------------------------------+-----------------+----------------------------------+-------------+---------------------+----------+
| id                                   | name            | tenant_id                        | vip_address | provisioning_status | provider |
+--------------------------------------+-----------------+----------------------------------+-------------+---------------------+----------+
| f6fe77ec-f6cf-47d7-adee-ed85454f05f0 | Load Balancer 1 | e4cb70dd38b44665bd20dce52e5c8c51 | 10.0.0.17   | ACTIVE              | haproxy  |
+--------------------------------------+-----------------+----------------------------------+-------------+---------------------+----------+
(neutron) lbaas-loadbalancer-delete f6fe77ec-f6cf-47d7-adee-ed85454f05f0
Deleted loadbalancer(s): f6fe77ec-f6cf-47d7-adee-ed85454f05f0



 

 

반응형

+ Recent posts