Revisiting proportion estimators

Abstract

Proportion estimators are quite frequently used in many application areas. The conventional proportion estimator (number of events divided by sample size) encounters a number of problems when the data are sparse as will be demonstrated in various settings. The problem of estimating its variance when sample sizes become small is rarely addressed in a satisfying framework. Specifically, we have in mind applications like the weighted risk difference in multicenter trials or stratifying risk ratio estimators (to adjust for potential confounders) in epidemiological studies. It is suggested to estimate p using the parametric family p̂cand p(1 - p) using p̂c(1 - p̂c), where p̂c= (X + c)/(n + 2c). We investigate the estimation problem of choosing c ≥ 0 from various perspectives including minimizing the average mean squared error of p̂c, average bias and average mean squared error of p̂c(1 - p̂c). The optimal value of c for minimizing the average mean squared error of p̂cis found to be independent of n and equals c = 1. The optimal value of c for minimizing the average mean squared error of p̂c(1 - p̂c) is found to be dependent of n with limiting value c = 0.833. This might justifiy to use a near-optimal value of c = 1 in practice which also turns out to be beneficial when constructing confidence intervals of the form p̂c± 1.96 √np̂c(1 - p̂c)/(n + 2c). © 2005 Edward Arnold (Publishers) Ltd.