SDGEN: Mimicking datasets for content generation in storage benchmarks

Raúl Gracia-Tinedo, Danny Harnik, Dalit Naor, Dmitry Sotnikov, Sivan Toledo, Aviad Zuck

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Storage system benchmarks either use samples of proprietary data or synthesize artificial data in simple ways (such as using zeros or random data). However, many storage systems behave completely differently on such artificial data than they do on real-world data. This is the case with systems that include data reduction techniques, such as compression and/or deduplication. To address this problem, we propose a benchmarking methodology called mimicking and apply it in the domain of data compression. Our methodology is based on characterizing the properties of real data that influence the performance of compressors. Then, we use these characterizations to generate new synthetic data that mimics the real one in many aspects of compression. Unlike current solutions that only address the compression ratio of data, mimicking is flexible enough to also emulate compression times and data heterogeneity. We show that these properties matter to the system’s performance. In our implementation, called SDGen, characterizations take at most 2.5KB per data chunk (e.g., 64KB) and can be used to efficiently share benchmarking data in a highly anonymized fashion; sharing it carries few or no privacy concerns. We evaluated our data generator’s accuracy on compressibility and compression times using real-world datasets and multiple compressors (lz4, zlib, bzip2 and lzma). As a proof-of-concept, we integrated SDGen as a content generation layer in two popular benchmarks (LinkBench and Impressions).

Original languageEnglish
Title of host publicationProceedings of the 13th USENIX Conference on File and Storage Technologies, FAST 2015
Pages317-330
Number of pages14
ISBN (Electronic)9781931971201
StatePublished - 2015
Event13th USENIX Conference on File and Storage Technologies, FAST 2015 - Santa Clara, United States
Duration: 16 Feb 201519 Feb 2015

Publication series

NameProceedings of the 13th USENIX Conference on File and Storage Technologies, FAST 2015

Conference

Conference13th USENIX Conference on File and Storage Technologies, FAST 2015
Country/TerritoryUnited States
CitySanta Clara
Period16/02/1519/02/15

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Networks and Communications
  • Software

Fingerprint

Dive into the research topics of 'SDGEN: Mimicking datasets for content generation in storage benchmarks'. Together they form a unique fingerprint.

Cite this