Selecting Sub-tables for Data Exploration

Kathy Razmadze, Yael Amsterdamer, Amit Somech, Susan B. Davidson, Tova Milo

Research output: Working paperPreprint

Abstract

We present a framework for creating small, informative sub-tables of large data tables to facilitate the first step of data science: data exploration. Given a large data table table T, the goal is to create a sub-table of small, fixed dimensions, by selecting a subset of T's rows and projecting them over a subset of T's columns. The question is: which rows and columns should be selected to yield an informative sub-table?
We formalize the notion of "informativeness" based on two complementary metrics: cell coverage, which measures how well the sub-table captures prominent association rules in T, and diversity. Since computing optimal sub-tables using these metrics is shown to be infeasible, we give an efficient algorithm which indirectly accounts for association rules using table embedding. The resulting framework can be used for visualizing the complete sub-table, as well as for displaying the results of queries over the sub-table, enabling the user to quickly understand the results and determine subsequent queries. Experimental results show that we can efficiently compute high-quality sub-tables as measured by our metrics, as well as by feedback from user-studies.
Original languageEnglish
Number of pages13
DOIs
StatePublished - 5 Mar 2022

Publication series

NameCoRR
PublisherCornell University Library, arXiv.org

Keywords

  • 68P15
  • Databases (cs.DB)
  • FOS: Computer and information sciences

Fingerprint

Dive into the research topics of 'Selecting Sub-tables for Data Exploration'. Together they form a unique fingerprint.

Cite this