TY - JOUR
T1 - Cramér-Rao Bound for Estimation after Model Selection and Its Application to Sparse Vector Estimation
AU - Meir, Elad
AU - Routtenberg, Tirza
N1 - Funding Information:
Manuscript received June 29, 2020; revised December 4, 2020 and February 14, 2021; accepted March 14, 2021. Date of publication March 24, 2021; date of current version April 20, 2021. This work was supported in part by the ISRAEL SCIENCE FOUNDATION (ISF) under Grant 1173/16, and in part by the BGU Cyber Security Research Center. (Corresponding author: Tirza Routtenberg.) The authors are with the School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel (e-mail: meirela@post.bgu.ac.il; tirzar@bgu.ac.il). Digital Object Identifier 10.1109/TSP.2021.3068356
Publisher Copyright:
© 1991-2012 IEEE.
PY - 2021/1/1
Y1 - 2021/1/1
N2 - In many practical parameter estimation problems, such as coefficient estimation of polynomial regression, the true model is unknown and thus, a model selection step is performed prior to estimation. The data-based model selection step affects the subsequent estimation. In particular, the oracle Cramér-Rao bound (CRB), which is based on knowledge of the true model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we investigate post-model-selection parameter estimation of a vector with an unknown support set, where this support set represents the model. We analyze the estimation performance of coherent estimators that force unselected parameters to zero. We use the mean-squared-selected-error (MSSE) criterion and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cramér-Rao-type bound on the MSSE and on the mean-squared-error (MSE) of any coherent estimator with a specific selective-bias function in the Lehmann sense. We implement the selective CRB for the special case of sparse vector estimation with an unknown support set. Finally, we demonstrate in simulations that the proposed selective CRB is an informative lower bound on the performance of the maximum selected likelihood estimator for a general linear model with the generalized information criterion and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the oracle CRB and Sando-Mitra-Stoica CRB (SMS-CRB) [1].
AB - In many practical parameter estimation problems, such as coefficient estimation of polynomial regression, the true model is unknown and thus, a model selection step is performed prior to estimation. The data-based model selection step affects the subsequent estimation. In particular, the oracle Cramér-Rao bound (CRB), which is based on knowledge of the true model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we investigate post-model-selection parameter estimation of a vector with an unknown support set, where this support set represents the model. We analyze the estimation performance of coherent estimators that force unselected parameters to zero. We use the mean-squared-selected-error (MSSE) criterion and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cramér-Rao-type bound on the MSSE and on the mean-squared-error (MSE) of any coherent estimator with a specific selective-bias function in the Lehmann sense. We implement the selective CRB for the special case of sparse vector estimation with an unknown support set. Finally, we demonstrate in simulations that the proposed selective CRB is an informative lower bound on the performance of the maximum selected likelihood estimator for a general linear model with the generalized information criterion and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the oracle CRB and Sando-Mitra-Stoica CRB (SMS-CRB) [1].
KW - Non-Bayesian selective estimation
KW - coherence estimation
KW - estimation after model selection
KW - selective Cramér-Rao bound
KW - sparse vector estimation
UR - http://www.scopus.com/inward/record.url?scp=85103296619&partnerID=8YFLogxK
U2 - 10.1109/TSP.2021.3068356
DO - 10.1109/TSP.2021.3068356
M3 - Article
AN - SCOPUS:85103296619
SN - 1053-587X
VL - 69
SP - 2284
EP - 2301
JO - IEEE Transactions on Signal Processing
JF - IEEE Transactions on Signal Processing
M1 - 9385866
ER -