## Abstract

Statistical divergences are ubiquitous in machine learning as tools for measuring discrepancy between probability distributions. As these applications inherently rely on approximating distributions from samples, we consider empirical approximation under two popular f-divergences: the total variation (TV) distance and the χ^{2}-divergence. To circumvent the sensitivity of these divergences to support mismatch, the framework of Gaussian smoothing is adopted. We study the limit distributions of n {δ (TV)(Pn} (N) σ,P∗(N) σ and n{2}(Pn} (N) σ } P∗(N) σ , where P_{n} is the empirical measure based on n independently and identically distributed (i.i.d.) observations from P, {{N) σ: = {{N)( {0,{σ 2}(I) d) \right), and ∗ stands for convolution. In arbitrary dimension, the limit distributions are characterized in terms of Gaussian process on ^{d} with covariance operator that depends on P and the isotropic Gaussian density of parameter σ. This, in turn, implies optimality of the n^{-1/2} expected value convergence rates recently derived for δ (TV )(Pn} (N) σ,P∗(N) σ and 2}(Pn} (N) σ P∗(N) σ . These strong statistical guarantees promote empirical approximation under Gaussian smoothing as a potent framework for learning and inference based on high-dimensional data.

Original language | English |
---|---|

Title of host publication | 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings |

Publisher | Institute of Electrical and Electronics Engineers |

Pages | 2640-2645 |

Number of pages | 6 |

ISBN (Electronic) | 9781728164328 |

DOIs | |

State | Published - 1 Jun 2020 |

Externally published | Yes |

Event | 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Los Angeles, United States Duration: 21 Jul 2020 → 26 Jul 2020 |

### Publication series

Name | IEEE International Symposium on Information Theory - Proceedings |
---|---|

Volume | 2020-June |

ISSN (Print) | 2157-8095 |

### Conference

Conference | 2020 IEEE International Symposium on Information Theory, ISIT 2020 |
---|---|

Country/Territory | United States |

City | Los Angeles |

Period | 21/07/20 → 26/07/20 |

## ASJC Scopus subject areas

- Theoretical Computer Science
- Information Systems
- Modeling and Simulation
- Applied Mathematics

## Fingerprint

Dive into the research topics of 'Limit Distributions for Smooth Total Variation and χ^{2}-Divergence in High Dimensions'. Together they form a unique fingerprint.