The Communication Complexity of Set Intersection Under Product Distributions

We consider a multiparty setting where k parties have private inputs X1, . . ., X_k ⊆ [n] and wish to compute the intersection^Tk_ℓ=1 X_ℓ of their sets, using as little communication as possible. This task generalizes the well-known problem of set disjointness, where the parties are required only to determine whether the intersection is empty or not. In the worst-case, it is known that the communication complexity of finding the intersection is the same as that of solving set disjointness, regardless of the size of the intersection: the cost of both problems is Ω (n log k + k) bits in the shared blackboard model, and Ω (nk) bits in the coordinator model. In this work we consider a realistic setting where the parties’ inputs are independent of one another, that is, the input is drawn from a product distribution. We show that this makes finding the intersection significantly easier than in the worst-case: only Θ̃((n^1−1/k (H(S) + 1)^1/k) + k) bits of communication are required, where H(S) is the Shannon entropy of the intersection S. We also show that the parties do not need to know the exact underlying input distribution; if we are given in advance O(n^1/k) samples from the underlying distribution µ, we can learn enough about µ to allow us to compute the intersection of an input drawn from µ using expected communication Θ̃((n^1−1/k E[|S|]^1/k) + k), where |S| is the size of the intersection.