Coded Retransmission in Wireless Networks Via Abstract MDPs: Theory and Algorithms

Mark Shifrin, Asaf Cohen, Olga Weisman, Omer Gurewitz

Research output: Contribution to journalArticlepeer-review


Consider a transmission scheme with a single transmitter and multiple receivers over a faulty broadcast channel. For each receiver, the transmitter has a unique infinite stream of packets, and its goal is to deliver them at the highest throughput possible. While such multiple-unicast models are unsolved in general, several network coding-based schemes were suggested. In such schemes, the transmitter can either send an uncoded packet, or a coded packet which is the function of a few packets. The packets sent can be received by the designated receiver (with some probability) or heard and stored by other receivers. Two functional modes are considered; the first presumes that the storage time is unlimited, while in the second one it is limited by the given time to expire (TTE) parameter. We model the transmission process as an infinite-horizon Markov decision process (MDP). Since the large state space renders exact solutions computationally impractical, we introduce policy-restricted and induced MDPs with significantly reduced state space, and prove that with proper reward function they have equal optimal value function (hence equal optimal throughput). We then derive a reinforcement learning algorithm, which learns the optimal policy for the induced MDP. This optimal strategy of the induced MDP, once applied to the policy of the restricted one, significantly improves over uncoded schemes. Next, we enhance the algorithm by means of analysis of the structural properties of the resulting cost functional. We demonstrate that our method scales well in the number of users, and automatically adapts to the packet loss rates, unknown in advance. In addition, the performance is compared to the recent bound by Wang, which assumes much stronger coding (e.g., intrasession and buffering of coded packets), yet is shown to be comparable.

Original languageEnglish
Article number7428954
Pages (from-to)4292-4306
Number of pages15
JournalIEEE Transactions on Wireless Communications
Issue number6
StatePublished - 1 Jun 2016


  • Markov Decision Processes
  • Network Coding
  • Reinforcement Learning

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


Dive into the research topics of 'Coded Retransmission in Wireless Networks Via Abstract MDPs: Theory and Algorithms'. Together they form a unique fingerprint.

Cite this