KIISE Transactions on Computing Practices (정보과학회 컴퓨팅의 실제 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Analysis of NVMe SSDs and Design of Direct Access Engine on Virtualized Environment

가상화 환경에서 NVMe SSD 성능 분석 및 직접 접근 엔진 개발

  • Received : 2017.10.19
  • Accepted : 2018.01.10
  • Published : 2018.03.15
⟨ Previous Next ⟩

Abstract

NVMe(Non-Volatile Memory Express) SSD(Solid State Drive) is a high-performance storage that makes use of flash memory as a storage cell, PCIe as an interface and NVMe as a protocol on the interface. It supports multiple I/O queues which makes it feasible to process parallel-I/Os on multi-core environments and to provide higher bandwidth than SATA SSDs. Hence, NVMe SSD is considered as a next generation-storage for data-center and cloud computing system. However, in the virtualization system, the performance of NVMe SSD is not fully utilized due to the bottleneck of the software I/O stack. Especially, when it uses I/O stack of the hypervisor or the host operating system like Xen and KVM, I/O performance degrades seriously due to doubled-I/O stack between host and virtual machine. In this paper, we propose a new I/O engine, called Direct-AIO (Direct-Asynchronous I/O) engine, that can access NVMe SSD directly for I/O performance improvements on QEMU emulator. We develop our proposed I/O engine and analyze I/O performance differences between the existed I/O engine and Direct-AIO engine.

낸드 플래시 메모리 기반의 NVMe(Non-Volatile Memory express) SSD(Solid State Drive)는 멀티 I/O 큐 환경을 제공하는 PCIe 인터페이스 기반에 NVMe 프로토콜을 사용하는 저장장치이다. NVMe SSD는 Multi-core 시스템에서 병렬 I/O 처리가 가능하고 SATA SSD에 비해 대역폭이 크며 대용량의 저장 공간을 제공하기 때문에 데이터 센터, 클라우드 컴퓨팅 등에 사용될 차세대 저장장치로 주목받고 있다. 하지만 가상화 시스템에서는 소프트웨어 I/O 스택의 병목으로 인하여 NVMe SSD의 성능을 충분히 활용하지 못하고 있다. 특히, Xen과 KVM과 같이 호스트 시스템의 I/O 스택을 사용하는 경우에는, 호스트 시스템과 가상머신의 중복된 I/O 스택을 통해서 입출력이 처리되기 때문에 성능 저하가 크게 발생한다. 본 논문에서는 NVMe SSD에 직접 접근하는 기술을 KVM 가상화 시스템에 적용함으로써 가상 머신 I/O의 성능을 향상시키는 Direct-AIO (Direct-Asynchronous I/O)엔진을 제안한다. 그리고 QEMU 에뮬레이터에 제안한 엔진을 개발하고 기존의 I/O 엔진과의 성능 차이를 비교 및 분석한다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. Intel(R) Virtualization Technology(Intel(R) VT), [Online] Available: http://www.intel.com/technology/virtualization/technology.htm
  2. AMD Virtualization Technology(AMD-V), [Online] Available: http://www.amd.com/en-us/solutions/servers/virtualization
  3. M. Ben-Yehuda, J. Mason, O. Krieger, J. Xenidis, L. Doorn, A. Mallick, J. Nakajima and E. Wahlig, "Utilizing IOMMUs for Virtualization in LInux and Xen," Proc. of the Ottawa Linux Symposium (OLS 2006).
  4. Y. Dong, X. Yang, J. Li, G. Liao, K. Tian and H. Guan, "High performance network virtualization with SR-IOV," Journal of Parallel and Distributed Computing, Vol. 72, Issue 11, pp. 1471-1480, Nov, 2012. https://doi.org/10.1016/j.jpdc.2012.01.020
  5. J. Shafer, "I/O virtualization bottlenecks in cloud computing today," Second Workshop on I/O Virtualization(WIOV) 2010.
  6. NVM Express [Online] Available: http://www.nvmexpress.org
  7. T. Kim, D. Kang, D. Lee and Y. Eom, "Improving Performance by Bridging the Semantic Gap between Multi-queue SSD and I/O Virtualization Framework," 31st International Conference on Massive Storage Systems and Technology (MSST 2015).
  8. M. Oh, H. Eom and H. Yeom, "Enhancing the I/O System for Virtual Machines Using High Performance SSDs," IEEE 33rd International Performance Computing and Communications Conference (IPCCC 2014).
  9. J. Kim, S. Ahn, K. La and W. Chang, "Improving I/O Performance of NVMe SSD on Virtual Machines," The 31st ACM/SIGAPP Symposium on Applied Computing (SAC 2016).
  10. A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori, "kvm: the Linux Virtual machine monitor," Proceedings of the Ottawa Linux Symposium (OLS 2007).
  11. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, A. Warfield, "Xen and the art of virtualization," Proc. of the nineteenth ACM symposium on Operating systems principles (SOSP), pp. 164-177.
  12. VirtualBox [Online] Available: https://www.virtualbox.org/
  13. J. Sugerman, G. Venkitachalam, B. Lim, "Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor," Proc. of the General Track: 2001 USENIX Annual Technical Conference (ATC), pp. 1-14, Jun. 25-30, 2001.
  14. I. Ahmad, "Easy and Efficient Disk I/O Workload Characterization in VMware ESX Server," 2007 IEEE 10th International Symposium on Workload Characterization (IISWC 07).
  15. H. Kim, Y. Lee and J. Kim, "NVMeDirect: A User-space I/O Framework for Application-specific Optimization on NVMe SSDs," Proc. of the 8th USENIX Conference on Hot Topics in Storage and File Systems 2016 (HotStorage 2016).
  16. Introduction to the Storage Performance Development Kit (SPDK) [Online]. Available: https://software.intel.com/en-us/articles/introduction-to-the-storage-performance-development-kit-spdk
  17. NVM Express 1.2 [Online]. Available: http://www.nvmexpress.org/specifications/
  18. FIO: flexible IO tester [Online]. Available: http://freecode.com/projects/fio
  19. M. Bjorling, J. Axboe, D. Nellans, P. Bonnet, "Linux block IO: introducing multi-queue SSD access on multi-core systems," Proc. of the 6th International Systems and Storage Conference, Jun. 30-Jul. 02, 2013.