Neural Estimation of Statistical Divergences: Theoretical Foundations and Applications to Communication Systems

Recent computational advances and an abundance of data have dramatically boosted reliability and per» formance of information processing systems, encompassing wireless communication, data compression, and machine learning. Many applications in those fields can be framed as quantifying or optimizing statistical divergences (SDs), which are measures of discrepancy between probability distributions. While there are many classic approaches for estimating SDs from data, neural estimators have become the method of choice when dealing with large, high—dimensional datasets. Their popularity stems from excellent scalability and computational efficiency observed in practice, but formal performance guarantees remain largely obscure. To facilitate principled implementations and unlock new application domains, this proposal puts forth an ambitious research agenda of neural estimation theory and practice. Specifically, this project will develop a comprehensive statistical and computational theory of neural estimation and leverage it towards novel applications to communications; far beyond the reach of classic approaches. The project will result in the underpinnings of neural estimation, that provide guidance and insight for practical implementations and significantly broaden the application domain of this powerful paradigm.

The proposed research is timely and has the potential to promote principled approaches to statistical learning based on a rigorous neural estimation theory. The proposed applications to communication systems are expected to have direct impact on critical infrastructure for industries like wireless communication and data storage. The project will also have broad impacts through a deliberate approach to education, training, and cross—institutional collaborations. Towards that end, the Pls will: (1) propose undergraduate research projects and develop a curriculum around neural estimation that will be integrated it into their classes; (2) provide training opportunities for graduate students, with a strong emphasis on diverse recruitment to increase women’s participation in STEM; and (3) nurture existing and foster new collaborative relationships between, Duke, Cornell, and Ben Gurion, by means of graduate student exchanges, joint group meetings, and semiannual workshops. The overall goal is to build a vibrant community of researchers from the three institutions working in synergy to advance theory and practice of neural estimation for communications.