Although the term well-behaved statistic often seems to be used in the scientific literature in somewhat the same way as is well-behaved in mathematics (that is, to mean "non-pathological"[1] [2]) it can also be assigned precise mathematical meaning, and in more than one way. In the former case, the meaning of this term will vary from context to context. In the latter case, the mathematical conditions can be used to derive classes of combinations of distributions with statistics which are well-behaved in each sense.
First Definition: The variance of a well-behaved statistical estimator is finite and one condition on its mean is that it is differentiable in the parameter being estimated.[3]
Second Definition: The statistic is monotonic, well-defined, and locally sufficient.[4]
More formally the conditions can be expressed in this way. is a statistic for that is a function of the sample, . For to be well-behaved we require:
: Condition 1
differentiable in , and the derivative satisfies:
: Condition 2
In order to derive the distribution law of the parameter T, compatible with
\boldsymbolx
\{z1,\ldots,zm\}
\{x1,\ldots,x
m | |
m\}\inakX |
\rho(x1,\ldots,xm)=s
akXm
akS
s
\{x1,\ldots,xm\}
\{\breve\theta1,\ldots,\breve\thetaN\}
\breve\thetaj=h-1(s,\breve
j, | |
z | |
1 |
\ldots,\breve
j) | |
z | |
m |
\{\breve
j,\ldots,\breve | |
z | |
1 |
j\} | |
z | |
m |
\{Z1,\ldots,Zm|S=s\}
xi
\{X1,\ldots,Xm|S=s\}
The remainder of the present article is mainly concerned with the context of data mining procedures applied to statistical inference and, in particular, to the group of computationally intensive procedure that have been called algorithmic inference.
See main article: Algorithmic inference.
In algorithmic inference, the property of a statistic that is of most relevance is the pivoting step which allows to transference of probability-considerations from the sample distribution to the distribution of the parameters representing the population distribution in such a way that the conclusion of this statistical inference step is compatible with the sample actually observed.
By default, capital letters (such as U, X) will denote random variables and small letters (u, x) their corresponding realizations and with gothic letters (such as
akU,akX
\boldsymbolx=\{x1,\ldots,xm\}
(g\theta,Z)
\theta
\boldsymbolx=\{g\theta(z1),\ldots,g\theta(zm)\}.
(g\theta,\boldsymbolz)
\{x1,\ldots,xm\}
akS
s=\rho(x1,\ldots,xm)=\rho(g\theta(z1),\ldots,g\theta(zm))=h(\theta,z1,\ldots,zm), (1)
for suitable seeds
\boldsymbolz=\{z1,\ldots,zm\}
For instance, for both the Bernoulli distribution with parameter p and the exponential distribution with parameter ? the statistic
m | |
\sum | |
i=1 |
xi
gp(u)=1
u\leqp
gλ(u)=-logu/λ
sp=\sum
m | |
i=1 |
I[0,p](ui)
s | ||||
|
m | |
\sum | |
i=1 |
logui.
Vice versa, in the case of X following a continuous uniform distribution on
[0,A]
\{c,c/2,c/3\}
s'A=11/6c
ga(u)=ua
sA=\sum
m | |
i=1 |
uia
\{0.8,0.8,0.8\}
\brevea=0.76c
sA=max\{x1,\ldots,xm\}
Analogously, for a random variable X following the Pareto distribution with parameters K and A (see Pareto example for more detail of this case),
s1=\sum
m | |
i=1 |
logxi
s2=mini=1,\ldots,m\{xi\}
As a general statement that holds under weak conditions, sufficient statistics are well-behaved with respect to the related parameters. The table below gives sufficient / Well-behaved statistics for the parameters of some of the most commonly used probability distributions.
Uniform discrete | f(x;n)=1/nI\{1,2,\ldots,n\ | sn=maxixi | ||||||||||||||||
Bernoulli | f(x;p)=px(1-p)1-xI\{0,1\ | sP=\sum
xi | ||||||||||||||||
Binomial | f(x;n,p)=\binom{n}{x}px(1-p)n-xI0,1,\ldots,(x) | sP=\sum
xi | ||||||||||||||||
Geometric | f(x;p)=p(1-p)xI\{0,1,\ldots\ | sP=\sum
xi | ||||||||||||||||
Poisson | f(x;\mu)=e-\mu\mux/x!I\{0,1,\ldots\ | sM
xi | ||||||||||||||||
Uniform continuous | f(x;a,b)=1/(b-a)I[a,b](x) | sA=minixi;sB=maxixi | ||||||||||||||||
Negative exponential | f(x;λ)=λe-λI[0,infty](x) | sΛ
xi | ||||||||||||||||
Pareto | f(x;a,k)=
\right)-aI[k,infty](x) | sA
logxi;sK=minixi | ||||||||||||||||
Gaussian | f(x,\mu,\sigma)=1/(\sqrt{2\pi}\sigma)
| sM=\sum
xi;s\Sigma
x)2} | ||||||||||||||||
Gamma | f(x;r,λ)=λ/\Gamma(r)(λx)r-1e-λI[0,infty](x) | sΛ
xi;sK
xi |