TY - JOUR
T1 - Interpretations of directed information in portfolio theory, data compression, and hypothesis testing
AU - Permuter, Haim H.
AU - Kim, Young Han
AU - Weissman, Tsachy
N1 - Funding Information:
Manuscript received December 24, 2009; revised November 24, 2010; accepted November 27, 2010. Date of current version May 25, 2011. This work was supported in part by NSF Grant CCF-0729195 and in part by BSF Grant 2008402. H. H. Permuter was supported in part by the Marie Curie Reintegration fellowship.
PY - 2011/6/1
Y1 - 2011/6/1
N2 - We investigate the role of directed information in portfolio theory, data compression, and statistics with causality constraints. In particular, we show that directed information is an upper bound on the increment in growth rates of optimal portfolios in a stock market due to causal side information. This upper bound is tight for gambling in a horse race, which is an extreme case of stock markets. Directed information also characterizes the value of causal side information in instantaneous compression and quantifies the benefit of causal inference in joint compression of two stochastic processes. In hypothesis testing, directed information evaluates the best error exponent for testing whether a random process Y causally influences another process X or not. These results lead to a natural interpretation of directed information I(Yn to Xn) as the amount of information that a random sequence Y n = (Y1,Y2, Yn) causally provides about another random sequence Xn = (X1,X 2,Xn). A new measure, directed lautum information, is also introduced and interpreted in portfolio theory, data compression, and hypothesis testing.
AB - We investigate the role of directed information in portfolio theory, data compression, and statistics with causality constraints. In particular, we show that directed information is an upper bound on the increment in growth rates of optimal portfolios in a stock market due to causal side information. This upper bound is tight for gambling in a horse race, which is an extreme case of stock markets. Directed information also characterizes the value of causal side information in instantaneous compression and quantifies the benefit of causal inference in joint compression of two stochastic processes. In hypothesis testing, directed information evaluates the best error exponent for testing whether a random process Y causally influences another process X or not. These results lead to a natural interpretation of directed information I(Yn to Xn) as the amount of information that a random sequence Y n = (Y1,Y2, Yn) causally provides about another random sequence Xn = (X1,X 2,Xn). A new measure, directed lautum information, is also introduced and interpreted in portfolio theory, data compression, and hypothesis testing.
KW - Causal conditioning
KW - Kelly gambling
KW - causal side information
KW - directed information
KW - hypothesis testing
KW - instantaneous compression
KW - lautum information
KW - portfolio theory
UR - http://www.scopus.com/inward/record.url?scp=79957641235&partnerID=8YFLogxK
U2 - 10.1109/TIT.2011.2136270
DO - 10.1109/TIT.2011.2136270
M3 - Article
AN - SCOPUS:79957641235
SN - 0018-9448
VL - 57
SP - 3248
EP - 3259
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
IS - 6
M1 - 5773045
ER -