Scaling concurrent log-structured data stores

Guy Golan-Gueta, Edward Bortnikov, Eshcar Hillel, Idit Keidar

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Log-structured data stores (LSM-DSs) are widely accepted as the state-of-the-art implementation of key-value stores. They replace random disk writes with sequential I/O, by accumulating large batches of updates in an in-memory data structure and merging it with the on-disk store in the background. While LSM-DS implementations proved to be highly successful at masking the I/O bottleneck, scaling them up on multicore CPUs remains a challenge. This is nontrivial due to their often rich APIs, as well as the need to coordinate the RAM access with the background I/O. We present cLSM, an algorithm for scalable concurrency in LSM-DS, which exploits multiprocessor-friendly data structures and non-blocking synchronization. cLSM supports a rich API, including consistent snapshot scans and general non-blocking read-modify-write operations. We implement cLSM based on the popular LevelDB keyvalue store, and evaluate it using intensive synthetic workloads as well as ones from production web-serving applications. Our algorithm outperforms state of the art LSM-DS implementations, improving throughput by 1.5x to 2.5x. Moreover, cLSM demonstrates superior scalability with the number of cores (successfully exploiting twice as many cores as the competition).

Original languageEnglish
Title of host publicationProceedings of the 10th European Conference on Computer Systems, EuroSys 2015
ISBN (Electronic)9781450332385
StatePublished - 17 Apr 2015
Event10th European Conference on Computer Systems, EuroSys 2015 - Bordeaux, France
Duration: 21 Apr 201524 Apr 2015

Publication series

NameProceedings of the 10th European Conference on Computer Systems, EuroSys 2015


Conference10th European Conference on Computer Systems, EuroSys 2015

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Information Systems
  • Computer Networks and Communications
  • Software
  • Control and Systems Engineering


Dive into the research topics of 'Scaling concurrent log-structured data stores'. Together they form a unique fingerprint.

Cite this