Geometric Sketch: The Inflatable-Shrinkable Sketch

A sketch is a probabilistic data-structure useful for collecting statistics about data streams, such as the frequency of each item. Sketches require sub-linear memory and can be updated and queried quickly. Their estimation errors are bounded to a known range with a certain probability, both determined by the sketch’s parameters, which allows users to tailor a sketch to their accuracy needs. Conventional sketches, such as count-sketch (CS) and count-min sketch (CMS) are configured with a fixed size, set during their initialization. That is, their parameters remain constant, while their absolute estimation error grows with the length of the stream. Unfortunately, this means that additional memory cannot be utilized to limit the estimation error buildup. Being able to dynamically increase and decrease the sketch’s memory usage is therefore attractive when the stream size is unknown in advance, or memory availability changes after initialization. To that end, we present Geometric Sketch (GS), a novel sketch that supports both increasing and decreasing memory usage at a granularity of a single counter, with a memory overhead of only 2 integers that are not counters. All of our code is open sourced [2].