Simultaneous Sensing and Channel Access based on Partial Observations via Deep Reinforcement Learning

Yoel Bokobza, Ron Dabora, Kobi Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper is eligible for the Jack Keil Wolf ISIT Student Paper Award. In this paper we study dynamic spectrum access (DSA) in cognitive wireless networks, consisting of primary users (PUs) and a secondary user (SU) which has only partial observations. The traffic patterns of the PUs are modeled as finite-memory Markov chains, and are unknown to the SU. It is noted that as observations are partial, then both channel sensing and channel access actions affect the throughput. Our objective in this work is to design a DSA algorithm such that the SU's long-term throughput is maximized. To that aim, we show theoretically that the DSA problem can be formulated as a single-agent problem with a single policy for both sensing and access, and propose a novel algorithm that learns both the optimal access policy and the optimal sensing policy via deep Q-learning, which is referred to as Double Deep Q-network for Sensing and Access (DDQSA). To the best of our knowledge, this is the first instance of a deep Q-learning-based DSA algorithm, which learns both sensing and access policies. Our results show that the DDQSA algorithm learns a policy that implements both sensing and channel access, and achieves significantly better performance compared to existing approaches.

Original languageEnglish
Title of host publication2022 IEEE International Symposium on Information Theory, ISIT 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2684-2689
Number of pages6
ISBN (Electronic)9781665421591
DOIs
StatePublished - 1 Jan 2022
Event2022 IEEE International Symposium on Information Theory, ISIT 2022 - Espoo, Finland
Duration: 26 Jun 20221 Jul 2022

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2022-June
ISSN (Print)2157-8095

Conference

Conference2022 IEEE International Symposium on Information Theory, ISIT 2022
Country/TerritoryFinland
CityEspoo
Period26/06/221/07/22

Keywords

  • Cognitive radio networks
  • deep reinforcement learning
  • dynamic spectrum access
  • wireless channels

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Simultaneous Sensing and Channel Access based on Partial Observations via Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this