Computable bounds for rate distortion with feed forward for stationary and ergodic sources

Iddo Naiss, Haim H. Permuter

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


In this paper, we consider the rate distortion problem of discrete-time, ergodic, and stationary sources with feed forward at the receiver. We derive a sequence of achievable and computable rates that converge to the feed-forward rate distortion. We show that for ergodic and stationary sources, the rate R-{n}(D)= {{1}\over {n}}\min I({\hat {X}}^{n} \rightarrow{} X^{n}) is achievable for any n, where the minimization is performed over the transition conditioning probability p({\hat {x}}^{n}\vert x^{n}) such that \BBE \left [{d(X^{n}, {\hat {X}}^{n})}\right]\leq D. We also show that the limit of R-{n}(D) exists and is the feed-forward rate distortion. We follow Gallager's proof where there is no feed forward and, with appropriate modification, obtain our result. We provide an algorithm for calculating R-{n}(D) using the alternating minimization procedure and present several numerical examples. We also present a dual form for the optimization of R-{n}(D) and transform it into a geometric programming problem.

Original languageEnglish
Article number6320694
Pages (from-to)760-781
Number of pages22
JournalIEEE Transactions on Information Theory
Issue number2
StatePublished - 24 Jan 2013


  • Alternating minimization procedure
  • Blahut-Arimoto (BA) algorithm
  • causal conditioning
  • concatenating code trees
  • directed information
  • ergodic and stationary sources
  • ergodic modes
  • geometric programming (GP)
  • rate distortion with feed forward

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Computable bounds for rate distortion with feed forward for stationary and ergodic sources'. Together they form a unique fingerprint.

Cite this