Limit Distributions for Smooth Total Variation and χ2-Divergence in High Dimensions

Ziv Goldfeld, Kengo Kato

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Statistical divergences are ubiquitous in machine learning as tools for measuring discrepancy between probability distributions. As these applications inherently rely on approximating distributions from samples, we consider empirical approximation under two popular f-divergences: the total variation (TV) distance and the χ2-divergence. To circumvent the sensitivity of these divergences to support mismatch, the framework of Gaussian smoothing is adopted. We study the limit distributions of n {δ (TV)(Pn} (N) σ,P∗(N) σ and n{2}(Pn} (N) σ } P∗(N) σ , where Pn is the empirical measure based on n independently and identically distributed (i.i.d.) observations from P, {{N) σ: = {{N)( {0,{σ 2}(I) d) \right), and ∗ stands for convolution. In arbitrary dimension, the limit distributions are characterized in terms of Gaussian process on d with covariance operator that depends on P and the isotropic Gaussian density of parameter σ. This, in turn, implies optimality of the n-1/2 expected value convergence rates recently derived for δ (TV )(Pn} (N) σ,P∗(N) σ and 2}(Pn} (N) σ P∗(N) σ . These strong statistical guarantees promote empirical approximation under Gaussian smoothing as a potent framework for learning and inference based on high-dimensional data.

Original languageEnglish
Title of host publication2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers
Pages2640-2645
Number of pages6
ISBN (Electronic)9781728164328
DOIs
StatePublished - 1 Jun 2020
Externally publishedYes
Event2020 IEEE International Symposium on Information Theory, ISIT 2020 - Los Angeles, United States
Duration: 21 Jul 202026 Jul 2020

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2020-June
ISSN (Print)2157-8095

Conference

Conference2020 IEEE International Symposium on Information Theory, ISIT 2020
Country/TerritoryUnited States
CityLos Angeles
Period21/07/2026/07/20

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Limit Distributions for Smooth Total Variation and χ2-Divergence in High Dimensions'. Together they form a unique fingerprint.

Cite this