Information efficiency

Joel Ratsaby

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Shannon's theory of information stands on a probabilistic representation of events that convey information, e.g., sending messages over a communication channel. Kolmogorov argues that information is a more fundamental concept which exists also in problems with no underlying stochastic model, for instance, the information contained in an algorithm or in the genome. In a classic paper he defines the discrete entropy of a finite set which establishes a combinatorial based definition of the information I(x : y) conveyed by a variable x (taking a binary string value x) about the unknown value of a variable y. The current paper extends Kolmogorov's definition of information to a more general setting where given 'x = x' there may still be uncertainty about the set of possible values of y. It then establishes a combinatorial based description complexity of x and introduces a novel concept termed information width, similar to n-widths in approximation theory. This forms the basis of new measures of cost and efficiency of information which give rise to a new framework whereby information of any input source, e.g., sample-based, general side-information or a hybrid of both, is represented and computed according to a single common formula. As an application, we consider the space of Boolean functions where input strings x correspond to descriptions of properties of classes of Boolean functions.

Original languageEnglish
Title of host publicationSOFSEM 2007
Subtitle of host publicationTheory and Practice of Computer Science - 33rd Conference on Current Trends in Theory and Practice of Computer Science, Proceedings
PublisherSpringer Verlag
Pages475-487
Number of pages13
ISBN (Print)9783540695066
DOIs
StatePublished - 1 Jan 2007
Event33rd Conference on Current Trends in Theory and Practice of Computer Science, SOFSEM 2007 - Harrachov, Czech Republic
Duration: 20 Jan 200726 Jan 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4362 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference33rd Conference on Current Trends in Theory and Practice of Computer Science, SOFSEM 2007
Country/TerritoryCzech Republic
CityHarrachov
Period20/01/0726/01/07

Fingerprint

Dive into the research topics of 'Information efficiency'. Together they form a unique fingerprint.

Cite this