Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks

Yoel Bokobza, Ron Dabora, Kobi Cohen

Research output: Working paper/PreprintPreprint

25 Downloads (Pure)

Abstract

We consider the problem of dynamic spectrum access (DSA) in cognitive wireless networks, where only partial observations are available to the users due to narrowband sensing and transmissions. The cognitive network consists of primary users (PUs) and a secondary user (SU), which operate in a time duplexing regime. The traffic pattern for each PU is assumed to be unknown to the SU and is modeled as a finite-memory Markov chain. Since observations are partial, then both channel sensing and access actions affect the throughput. The objective is to maximize the SU's long-term throughput. To achieve this goal, we develop a novel algorithm that learns both access and sensing policies via deep Q-learning, dubbed Double Deep Q-network for Sensing and Access (DDQSA). To the best of our knowledge, this is the first paper that solves both sensing and access policies for DSA via deep Q-learning. Second, we analyze the optimal policy theoretically to validate the performance of DDQSA. Although the general DSA problem is P-SPACE hard, we derive the optimal policy explicitly for a common model of a cyclic user dynamics. Our results show that DDQSA learns a policy that implements both sensing and channel access, and significantly outperforms existing approaches.
Original languageEnglish GB
StatePublished - 1 Oct 2021

Keywords

  • Computer Science - Information Theory
  • Computer Science - Machine Learning

Fingerprint

Dive into the research topics of 'Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks'. Together they form a unique fingerprint.

Cite this