Design and Evaluation of an Efficient Flushing Scheme for key-value Store

키-값 저장소를 위한 효율적인 로그 처리 기법 설계 및 평가

  • 한혁 (동덕여자대학교 컴퓨터학과)
  • Received : 2019.02.08
  • Accepted : 2019.04.30
  • Published : 2019.05.28


Key-value storage engines are an essential component of growing demand in many computing environments, including social networks, online e-commerce, and cloud services. Recent key-value storage engines offer many features such as transaction, versioning, and replication. In a key-value storage engine, transaction processing provides atomicity through Write-Ahead-Logging (WAL), and a synchronous commit method for transaction processing flushes log data before the transaction completes. According to our observation, flushing log data to persistent storage is a performance bottleneck for key-value storage engines due to the significant overhead of fsync() calls despite the various optimizations of existing systems. In this article, we propose a group synchronization method to improve the performance of the key-value storage engine. We also design and implement a transaction scheduling method to perform other transactions while the system processes fsync() calls. The proposed method is an efficient way to reduce the number of frequent fsync() calls in the synchronous commit while supporting the same level of transaction provided by the existing system. We implement our scheme on the WiredTiger storage engine and our experimental results show that the proposed system improves the performance of key-value workloads over existing systems.


Key-value Storage Engine;Write-ahead Logging;Synchronization

CCTHCV_2019_v19n5_187_f0001.png 이미지

그림 1. WiredTiger의 통합 어레이 기법 (Tx : 트랜잭션, LR :로그 레코드, sync lock : 동기화 잠금)

CCTHCV_2019_v19n5_187_f0002.png 이미지

그림 2. YCSB 성능 결과

CCTHCV_2019_v19n5_187_f0003.png 이미지

그림 3. YCSB 수행 시간 분석

CCTHCV_2019_v19n5_187_f0004.png 이미지

그림 4. 통합 동기화 기법 (T : 쓰레드)

CCTHCV_2019_v19n5_187_f0005.png 이미지

그림 5. 최적화된 스케줄링 기법

CCTHCV_2019_v19n5_187_f0006.png 이미지

그림 6. YCSB 성능 평가 결과

CCTHCV_2019_v19n5_187_f0007.png 이미지

그림 7. OLTP 성능 평가 결과


Supported by : 동덕여자대학교


  1. Berk Atikoglu, Yuehai Xu, Eitan Frachtenberg, Song Jiang, and Mike Paleczny, "Workload analysis of a large-scale key-value store," In Proceedings of the 12th ACM international conference on Measurement and Modeling of Computer Systems (SIGMETRICS '12), 2012.
  2. Swaminathan Sivasubramanian, "Amazon dynamodb: a seamlessly scalable non-relational database service," In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (SIGMOD '12), 2012.
  3. J. L. Carlson, Redis in Action, Manning Publications Co., 2013.
  4. Brad Fitzpatrick, Distributed caching with memcached, Linux Journal, 2004.
  5. Roshan Sumbaly, Jay Kreps, Lei Gao, Alex Feinberg, Chinmay Soman, and Sam Shah, "Serving large-scale batch computed data with project Voldemort," In Proceedings of the 10th USENIX conference on File and Storage Technologies (FAST '12), 2012.
  6. Jing Han, E. Haihong, Guan Le, and Jian Du, "Survey on NoSQL database," In Proceedings of the 6th International Conference on Pervasive Computing and Applications, 2011.
  7. WiredTiger,, 2014.
  8. C. Mohan, Don Haderle, Bruce Lindsay, Hamid Pirahesh, and Peter Schwarz, "ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging," ACM Trans. Database Syst., Vol.17, No.1, pp.94-162, 1992.
  9. P. Helland, H. Sammer, J. Lyon, R. Carr, P. Garrett, and A. Reuter, "Group commit timers and high volume transaction systems," In Proceedings of High Performance Transaction Systems (HPTS), 1987.
  10. R. Ramakrishnan and J. Gehrke, Database management systems, Osborne/McGraw-Hill, 2000.
  11. Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis, and Anastasia Ailamaki, "Aether: a scalable approach to logging," In Proceedings of the 2010 international conference on very large database (VLDB '10), 2010.
  12. Jian Huang, Karsten Schwan, and Moinuddin K. Qureshi, "NVRAM-aware logging in transaction systems," Proceedings of the 2014 international conference on very large database (VLDB '14), 2014.
  13. Shimin Chen, "FlashLogging: exploiting flash devices for synchronous logging performance," In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data (SIGMOD '09), 2009.
  14. S. Gao, J. Xu, T. Harder, B. He, B. Choi, and H. Hu, "PCMLogging: Optimizing Transaction Logging and Recovery Performance with PCM," IEEE Transactions on Knowledge and Data Engineering, Vol.27, No.12, 2015.
  15. Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears, "Benchmarking cloud serving systems with YCSB," In Proceedings of the 1st ACM symposium on Cloud computing (SoCC '10), 2010.