Maximizing Efficiency with Complete Statistics

Complete statistics

The many choices with minimal sufficient statistics sometimes confuse a selection. This section introduces complete statistics, which narrows the pooling statistic choice to only the maximum likelihood estimator (MLE) of the feature map distribution.

A complete statistic is a bridge between minimal sufficient statistics and MLEsMaximum likelihood estimators. MLEs derived from complete minimal statistics have the essential attributes of unbiasedness and minimum variance along with the minimality and completeness properties. MLEs, therefore, become the natural choice for pooling. This removes most of the ambiguity around pooling statistic selection.

Next, we lay out the attributes and path leading to the relationship between complete minimal statistics and the MLE.

Completeness

Let $f(t|θ)$ be a family of pdfs or pmfs for a statistic $T(X)$. The family of probability distributions is called complete if for every measurable, real-valued function $g, E_{\theta}(g(T)) = 0$ for all $\theta ∈ Ω$ implies $g(T) = 0$ with respect to $\theta$, that is, $P_{\theta}(g(T) = 0) = 1$ for all $\theta$. The statistic $T$ is boundedly complete if $g$ is bounded.

In simple words, it means a probability distribution is complete if the probability of a statistic $T(X)$ from an observed sample $X = X_1,\ldots, X_n$ in the distribution is always non-zero.

It becomes clearer by considering a discrete case. In this case, completeness means $E_{\theta}(g(T)) = \sum g(T)P_{\theta}(T = t) = 0$ implies $g(T) = 0$ because by definition $P_{\theta}(T = t)$ is non-zero.

For example, suppose $X_1, \ldots , X_n$ is observed from a normal distribution $N(μ, 1)$, and there is a statistic $T(X) = \sum X_i$. Then, the $P_μ(T(X) = 0)$ is not equal to $0$ for all $μ$. Therefore, $E_μ(g(T)) = \int g(T)P_μ(T) = 0$ implies $g(T) = 0$ for all $μ$. Therefore, $T = \sum X_i$ is complete.

This is an important property because it confirms that a statistic $T$, if complete, will span the whole sample space. The statistic will contain some information from every observed sample $X_i$ ...