반응형
Kolla-ansible Openstack vGPU(Virtual GPU) 설정 방법
Kolla-ansible Openstack vGPU 사용을 위한 Openstack Host 환경 설정을 한다.
- Openstack vGPU Host 환경 세팅
1. BIOS 세팅
| ### BIOS 기본 세팅 ### VT-d on VT-x on |
2. Host OS 설치
| ### 지원 되는 OS 설치 ### o 본 설정에 맞는 OS를 설치한다. o Ubuntu, Rocky, CentOS 등 o 설치 버전 : Rocky 8 |
3. Host OS 환경 설정
| ### NVIDIA 그래픽 드라이버 인식 확인 [root@gpu01]# lspci -nn |grep NVIDIA c1:00.0 3D controller [0101]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1eb8] (rev a1) [root@gpu01]# cat /etc/default/grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="crashkernel=auto resume=/dev/mapper/rl-swap rd.lvm.lv=rl/root rd.lvm.lv=rl/swap intel_iommu=on vfio-pci.ids=10de:1eb8" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true [root@gpu01]# grub2-mkconfig -o /boot/efi/EFI/rocky/grub.cfg [root@gpu01]# mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak [root@gpu01]# dracut -v /boot/initramfs-$(uname -r).img $(uname -r) |
5. 서버 재기동
| ### blacklist 내용 추가 [root@gpu01]# vim /etc/modprobe.d/blacklist.conf blacklist nouveau ### 서버 재기동 [root@gpu01]# reboot [root@gpu01]# lsmod | grep nouveau -> 출력값 X |
6. NVIDIA 패키지 설치
| ### NVIDIA VGPU 패키지 설치를 위한 사전 작업 [root@gpu01]# dnf install make gcc -y [root@gpu01]# yum group install "Development Tools" ### 필요시 설치 [root@gpu01]# dnf install gcc* ### NVIDIA VGPU 설치시 해당 설정 요청에 따른 추가 [root@gpu01]# cat nvidia-installer-disable-nouveau.conf # generated by nvidia-installer blacklist nouveau options nouveau modeset=0 ### NVIDIA 패키지 공홈 다운 및 설치 [root@gpu01]# ./NVIDIA-Linux-x86_64-510.73.06-vgpu-kvm.run [root@gpu01]# nvidia-smi [root@gpu01]# reboot ← reboot 해야 mdev_bus 가 보인다 [root@gpu01]# ls /sys/class/mdev_bus/*/mdev_supported_types nvidia-222 nvidia-224 nvidia-226 nvidia-228 nvidia-230 nvidia-232 nvidia-234 nvidia-319 nvidia-321 nvidia-223 nvidia-225 nvidia-227 nvidia-229 nvidia-231 nvidia-233 nvidia-252 nvidia-320 ### 위 폴더 및 리스트가 뜨면 확인 VGPU 기본 환경 설정 완료. |
7. Openstack Config 수정
Docker container 기반 Openstack Kolla-ansible 설치
| #################################################################################### ### Contoller 서버 nova-api, nova.conf 설정값 추가 [pci] alias = { "vendor_id":"10de", "product_id":"1eb8", "device_type":"type-PF", "name":"T4" } passthrough_whitelist = { "vendor_id":"10de", "product_id":"1eb8" } ### controller 서버 nova-scheduler nova.conf 설정 추가 [filter_scheduler] enabled_filters = RetryFilter,AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter,PciPassthroughFilter available_filters = nova.scheduler.filters.all_filters #################################################################################### ### Nova Compute Node nova.conf 설정 [devices] enabled_vgpu_types = nvidia-223 #################################################################################### [root@gpu01]# docker restart nova_compute |
| Graphic Card : Tesla V100S - Openstack 설정값 vGPU 사양 설명 |
| 1 nvidia-355 num_heads=4, frl_config=60, framebuffer=1024M, max_resolution=5120x2880, max_instance=32 2 nvidia-356 num_heads=4, frl_config=60, framebuffer=2048M, max_resolution=7680x4320, max_instance=16 3 nvidia-357 num_heads=4, frl_config=60, framebuffer=4096M, max_resolution=7680x4320, max_instance=8 4 nvidia-358 num_heads=4, frl_config=60, framebuffer=8192M, max_resolution=7680x4320, max_instance=4 5 nvidia-359 num_heads=4, frl_config=60, framebuffer=16384M, max_resolution=7680x4320, max_instance=2 6 nvidia-360 num_heads=4, frl_config=60, framebuffer=32768M, max_resolution=7680x4320, max_instance=1 7 nvidia-361 num_heads=1, frl_config=60, framebuffer=4096M, max_resolution=4096x2160, max_instance=8 8 nvidia-362 num_heads=1, frl_config=60, framebuffer=8192M, max_resolution=4096x2160, max_instance=4 9 nvidia-363 num_heads=1, frl_config=60, framebuffer=16384M, max_resolution=4096x2160, max_instance=2 10 nvidia-364 num_heads=1, frl_config=60, framebuffer=32768M, max_resolution=4096x2160, max_instance=1 11 nvidia-365 num_heads=1, frl_config=60, framebuffer=1024M, max_resolution=1280x1024, max_instance=32 12 nvidia-366 num_heads=1, frl_config=60, framebuffer=2048M, max_resolution=1280x1024, max_instance=16 13 nvidia-367 num_heads=1, frl_config=60, framebuffer=4096M, max_resolution=1280x1024, max_instance=8 14 nvidia-368 num_heads=1, frl_config=60, framebuffer=8192M, max_resolution=1280x1024, max_instance=4 15 nvidia-369 num_heads=1, frl_config=60, framebuffer=16384M, max_resolution=1280x1024, max_instance=2 16 nvidia-370 num_heads=1, frl_config=60, framebuffer=32768M, max_resolution=1280x1024, max_instance=1 17 nvidia-371 num_heads=4, frl_config=45, framebuffer=1024M, max_resolution=5120x2880, max_instance=32 18 nvidia-373 num_heads=4, frl_config=45, framebuffer=2048M, max_resolution=5120x2880, max_instance=16 |
| Graphic Card : Tesla T4 - Openstack 설정값 vGPU 사양 설명 |
| 1 nvidia-222 num_heads=4, frl_config=45, framebuffer=1024M, max_resolution=5120x2880, max_instance=16 2 nvidia-223 num_heads=4, frl_config=45, framebuffer=2048M, max_resolution=5120x2880, max_instance=8 3 nvidia-224 num_heads=4, frl_config=45, framebuffer=2048M, max_resolution=5120x2880, max_instance=8 4 nvidia-225 num_heads=1, frl_config=60, framebuffer=1024M, max_resolution=1280x1024, max_instance=16 5 nvidia-226 num_heads=1, frl_config=60, framebuffer=2048M, max_resolution=1280x1024, max_instance=8 6 nvidia-227 num_heads=1, frl_config=60, framebuffer=4096M, max_resolution=1280x1024, max_instance=4 7 nvidia-228 num_heads=1, frl_config=60, framebuffer=8192M, max_resolution=1280x1024, max_instance=2 8 nvidia-229 num_heads=1, frl_config=60, framebuffer=16384M, max_resolution=1280x1024, max_instance=1 9 nvidia-230 num_heads=4, frl_config=60, framebuffer=1024M, max_resolution=5120x2880, max_instance=16 10 nvidia-231 num_heads=4, frl_config=60, framebuffer=2048M, max_resolution=7680x4320, max_instance=8 11 nvidia-232 num_heads=4, frl_config=60, framebuffer=4096M, max_resolution=7680x4320, max_instance=4 12 nvidia-233 num_heads=4, frl_config=60, framebuffer=8192M, max_resolution=7680x4320, max_instance=2 13 nvidia-234 num_heads=4, frl_config=60, framebuffer=16384M, max_resolution=7680x4320, max_instance=1 14 nvidia-252 num_heads=4, frl_config=45, framebuffer=1024M, max_resolution=5120x2880, max_instance=16 15 nvidia-319 num_heads=1, frl_config=60, framebuffer=4096M, max_resolution=4096x2160, max_instance=4 16 nvidia-320 num_heads=1, frl_config=60, framebuffer=8192M, max_resolution=4096x2160, max_instance=2 17 nvidia-321 num_heads=1, frl_config=60, framebuffer=16384M, max_resolution=4096x2160, max_instance=1 |
8. Openstack Flavor 생성
| ### vGPU Flavor 생성, 설정 [root@controller01]# openstack flavor create --vcpus 2 --ram 2048 --disk 100 vgputest01 [root@controller01]# openstack flavor set vgputest01 --property "resources:VGPU=1"
|
GPU & vGPU 참고 사이트
- NVIDIA : Virtual GPU Software User Guide :: NVIDIA Virtual GPU Software Documentation
- vGPU : OpenStack Docs: Attaching virtual GPU devices to guests
- Supported NVIDIA GPUs: NVIDIA A10, A16, A30, A40, A100, T4, RTX 8000, RTX 6000, V100S, V100, P100, P40, P6, P4, M60, M10
- vGPU는 설치, 할당 가능하지만 NVIDIA 관련 라이센스 구매해야됨.
반응형
'Openstack > vGPU & GPU' 카테고리의 다른 글
| Kolla-ansible Openstack vGPU Virtual Machine 설정 방법 (0) | 2023.12.20 |
|---|---|
| Openstack GPU & VGPU 설정 방법 - Ubuntu (0) | 2023.11.27 |
| Openstack Kolla-ansible GPU & VGPU 세팅 (0) | 2023.08.10 |
