## Abstract

Pinsker's widely used inequality upper-bounds the total variation distance ||P - Q||_{1} in terms of the Kullback-Leibler divergence D(P||Q). Although, in general, a bound in the reverse direction is impossible, in many applications the quantity of interest is actually D*(v, Q) - defined, for an arbitrary fixed Q, as the infimum of D(P||Q) over all distributions P that are at least v-far away from Q in total variation. We show that D*(v, Q) ≤ Cv^{2} + O(v^{3}), where C = C(Q) = 1/2 for balanced distributions, thereby providing a kind of reverse Pinsker inequality. Some of the structural results obtained in the course of the proof may be of independent interest. An application to large deviations is given.

Original language | English |
---|---|

Article number | 6746175 |

Pages (from-to) | 3172-3177 |

Number of pages | 6 |

Journal | IEEE Transactions on Information Theory |

Volume | 60 |

Issue number | 6 |

DOIs | |

State | Published - 1 Jan 2014 |

## Keywords

- McDiarmid's inequality
- Pinsker's inequality
- Sanov's theorem

## ASJC Scopus subject areas

- Information Systems
- Computer Science Applications
- Library and Information Sciences

## Fingerprint

Dive into the research topics of 'Minimum KL-divergence on complements of L_{1}balls'. Together they form a unique fingerprint.