Network Partitioning and Avoidable Contention

Network contention frequently dominates the run time of parallel algorithms and limits scaling performance. Most previous studies mitigate or eliminate contention by utilizing one of several approaches: communication-minimizing algorithms; hotspot-avoiding routing schemes; topology-aware task mapping; or improving global network properties, such as bisection bandwidth, edge-expansion, partitioning, and network diameter. In practice, parallel jobs often use only a fraction of a host system. How do processor allocation policies affect contention within a partition? We utilize edge-isoperimetric analysis of network graphs to determine whether a network partition defined by a processor allocation has optimal internal bisection. Increasing the bisection allows a more efficient use of the network resources, decreasing or completely eliminating the link contention. We study torus networks and characterize partition geometries that maximize internal bisection bandwidth, and examine the allocation policies of Mira and JUQUEEN, the two largest publicly-accessible Blue∼Gene/Q torus-based supercomputers. Our analysis shows that the bisection bandwidth of their partitions can often be improved by changing the partitions' geometries, yielding up to a x2∼speedup for contention-bound workloads. Benchmark experiments validate the predictions. Our analysis applies to allocation policies of other networks.