Directed information, causal estimation, and communication in continuous time

Tsachy Weissman, Young Han Kim, Haim H. Permuter

Research output: Contribution to journalArticlepeer-review

29 Scopus citations


A notion of directed information between two continuous-time processes is proposed. A key component in the definition is taking an infimum over all possible partitions of the time interval, which plays a role no less significant than the supremum over 'space' partitions inherent in the definition of mutual information. Properties and operational interpretations in estimation and communication are then established for the proposed notion of directed information. For the continuous-time additive white Gaussian noise channel, it is shown that Duncan's classical relationship between causal estimation error and mutual information continues to hold in the presence of feedback upon replacing mutual information by directed information. A parallel result is established for the Poisson channel. The utility of this relationship is demonstrated in computing the directed information rate between the input and output processes of a continuous-time Poisson channel with feedback, where the channel input process is constrained to be constant between events at the channel output. Finally, the capacity of a wide class of continuous-time channels with feedback is established via directed information, characterizing the fundamental limit on reliable communication.

Original languageEnglish
Article number6353588
Pages (from-to)1271-1287
Number of pages17
JournalIEEE Transactions on Information Theory
Issue number3
StatePublished - 20 Feb 2013


  • Causal estimation
  • Duncan's theorem
  • Gaussian channel
  • Poisson channel
  • conditional mutual information
  • continuous time
  • directed information
  • feedback capacity
  • time partition

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Directed information, causal estimation, and communication in continuous time'. Together they form a unique fingerprint.

Cite this