Using measurements with large round-off errors for interval estimation of normal process variance

Benson Karhi Diamanta, Dvir Harcabi Ellite, Regev Itai, Schechtman Edna

Research output: Contribution to journalArticlepeer-review

Abstract

Large round-off errors may affect efforts to estimate the distribution parameters. The ratio between the standard deviation σ and the scale step h, δ = σ/h, of the measurement instrument, for which rounding off is large when δ < 0.5, determines the significance of the round off. In this study the authors present a new variance interval estimator based on the method of moments (MoM) approach using the bootstrap technique. The authors compare the MoM interval estimator with two a-parametric estimators, the naïve estimator and Sheppard's correction, using simulation. They find that the MoM interval estimator performs better than the a-parametric estimators in terms of coverage probability and interval length, especially for medium and large samples. The MoM interval estimator should be used to compensate for the large round off errors that can occur when using measurement instruments whose scale step is too large.

Original languageEnglish
Pages (from-to)1050-1056
Number of pages7
JournalIET Science, Measurement and Technology
Volume9
Issue number8
DOIs
StatePublished - 1 Nov 2015

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Using measurements with large round-off errors for interval estimation of normal process variance'. Together they form a unique fingerprint.

Cite this