Extension of the Blahut-Arimoto algorithm for maximizing directed information

Iddo Naiss, Haim H. Permuter

Research output: Contribution to journalArticlepeer-review

23 Scopus citations


In this paper, we extend the Blahut-Arimoto algorithm for maximizing Massey's directed information. The algorithm can be used for estimating the capacity of channels with delayed feedback, where the feedback is a deterministic function of the output. In order to maximize the directed information, we apply the ideas from the regular Blahut-Arimoto algorithm, i.e., the alternating maximization procedure, to our new problem. We provide both upper and lower bound sequences that converge to the optimum global value. Our main insight in this paper is that in order to find the maximum of the directed information over a causal conditioning probability mass function, one can use a backward index time maximization combined with the alternating maximization procedure. We give a detailed description of the algorithm, showing its complexity and the memory needed, and present several numerical examples.

Original languageEnglish
Article number6298962
Pages (from-to)204-222
Number of pages19
JournalIEEE Transactions on Information Theory
Issue number1
StatePublished - 7 Jan 2013


  • Alternating maximization procedure
  • Blahut-Arimoto algorithm
  • Ising channel
  • backward index time maximization
  • causal conditioning
  • channels with feedback
  • directed information
  • finite-state channels (FSCs)
  • trapdoor channel

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Extension of the Blahut-Arimoto algorithm for maximizing directed information'. Together they form a unique fingerprint.

Cite this