Subgradient Descent Learning Over Fading Multiple Access Channels With Over-the-Air Computation

Tamir L.S. Gez, Kobi Cohen

Research output: Contribution to journalArticlepeer-review

Abstract

We focus on a distributed learning problem in a communication network, consisting of N distributed nodes and a central parameter server (PS). The PS is responsible for performing the computation based on data received from the nodes, which are transmitted over a multiple access channel (MAC). The objective function for this problem is the sum of the local loss functions of the nodes. This problem has gained attention in the field of distributed sensing systems, as well as in the area of federated learning (FL) recently. However, current approaches to solving this problem rely on the assumption that the loss functions are continuously differentiable. In this paper, we first address the case where this assumption does not hold. We develop a novel algorithm called Sub-Gradient descent Multiple Access (SGMA) to solve the learning problem over MAC. SGMA involves each node transmitting an analog shaped waveform of its local subgradient over MAC, and the PS receiving a superposition of the noisy analog signals, resulting in a bandwidth-efficient over-the-air (OTA) computation used to update the learned model. We analyze the performance of SGMA and prove that it has a convergence rate that approaches that of the centralized subgradient algorithm in large networks. Simulation results using real datasets show the effectiveness of SGMA.

Original languageAmerican English
Pages (from-to)94623-94635
Number of pages13
JournalIEEE Access
Volume11
DOIs
StatePublished - 1 Jan 2023

Keywords

  • Distributed learning
  • federated learning (FL)
  • gradient descent (GD)-type learning
  • multiple access channel (MAC)
  • over-the-air (OTA) computation
  • subgradient methods

All Science Journal Classification (ASJC) codes

  • General Engineering
  • General Computer Science
  • General Materials Science

Fingerprint

Dive into the research topics of 'Subgradient Descent Learning Over Fading Multiple Access Channels With Over-the-Air Computation'. Together they form a unique fingerprint.

Cite this