KV-Direct: High-Performance In-Memory Key-Value Store with Programmable NIC
Performance of in-memory key-value store (KVS) continues to be of great importance as modern KVS goes beyond the traditional object-caching workload and becomes a key infrastructure to support distributed main-memory computation in data centers. Recent years have witnessed a rapid increase of network bandwidth in data centers, shifting the bottleneck of most KVS from the network to the CPU. RDMA-capable NIC partly alleviates the problem, but the primitives provided by RDMA abstraction are rather limited. Meanwhile, programmable NICs become available in data centers, enabling in-network processing. In this paper, we present KV-Direct, a high performance KVS that leverages programmable NIC to extend RDMA primitives and enable remote direct key-value access to the main host memory.
We develop several novel techniques to maximize the throughput and hide the latency of the PCIe connection between the NIC and the host memory, which becomes the new bottleneck. Combined, these mechanisms allow a single NIC KV-Direct to achieve up to 180 M key-value operations per second, equivalent to the throughput of tens of CPU cores. Compared with CPU based KVS implementation, KV-Direct improves power efficiency by 3x, while keeping tail latency below 10 µs. Moreover, KV-Direct can achieve near linear scalability with multiple NICs. With 10 programmable NIC cards in a commodity server, we achieve 1.22 billion KV operations per second, which is almost an order-of-magnitude improvement over existing systems, setting a new milestone for a general-purpose in-memory key-value store.
Bojie Li, Zhenyuan Ruan, Wencong Xiao, Yuanwei Lu, Yongqiang Xiong, Andrew Putnam, Enhong Chen and Lintao Zhang.
Proceedings of the 26th Symposium on Operating Systems Principles (SOSP ‘17). [PDF] [Slides]
- Oct. 2017, SOSP’17 Conference. [Slides] [Talk Transcription]
- Nov. 2017, Fudan University, Shanghai
- Nov. 2017, University of Science and Technology of China, Hefei
- Nov. 2017, Baidu Corporation, Beijing
- Dec. 2017, Open Source Operating System Annual Technical Conference 2017 (OS2ATC ‘17), Beijing
- Oct. 2017, SOSP’17 Student Travel Grant.
- Oct. 2017, Microsoft Research Asia Fellowship Award. [English Website] [Chinese Website]
- Oct. 2017, China National Ph.D. Scholarship.
- The morning paper: KV-Direct: High-performance in-memory key-value store with programmable NIC
- Microsoft Research Asia: 李博杰：折腾系统，让它的性能提升10倍
- Microsoft Research Asia: SOSP：计算机系统研究的风向标
- USTCIF 中国科学技术大学新创公益基金会: 阮震元：科大本科生顶会零突破
- 一个高性能K-V存储服务 K-V Direct
- Bojie Li, 3rd year Ph.D. student in MSRA and USTC
- Zhenyuan Ruan, 4th year undergraduine in USTC (now a Ph.D. student in UCLA)
- Wencong Xiao, 3rd year Ph.D. student in MSRA and Beihang University
- Yuanwei Lu, 4th year Ph.D. student in MSRA and USTC
- Dr. Yongqiang Xiong, Lead Researcher in Microsoft Research Asia
- Dr. Andrew Putnam, Principal Research Hardware Design Engineer in the Microsoft Research Technologies (MSR-T)
- Prof. Enhong Chen, Professor in USTC
- Dr. Lintao Zhang, Principal Researcher in Microsoft Research Asia