In statistics, a confidence region is a multi-dimensional generalization of a confidence interval. It is a set of points in an n-dimensional space, often represented as an ellipsoid around a point which is an estimated solution to a problem, although other shapes can occur.
The confidence region is calculated in such a way that if a set of measurements were repeated many times and a confidence region calculated in the same way on each set of measurements, then a certain percentage of the time (e.g. 95%) the confidence region would include the point representing the "true" values of the set of variables being estimated. However, unless certain assumptions about prior probabilities are made, it does not mean, when one confidence region has been calculated, that there is a 95% probability that the "true" values lie inside the region, since we do not assume any particular probability distribution of the "true" values and we may or may not have other information about where they are likely to lie.
Suppose we have found a solution
\boldsymbol{\beta}
Y=X\boldsymbol{\beta}+\boldsymbol{\varepsilon}
where Y is an n-dimensional column vector containing observed values of the dependent variable, X is an n-by-p matrix of observed values of independent variables (which can represent a physical model) which is assumed to be known exactly,
\boldsymbol{\beta}
\boldsymbol{\varepsilon}
\sigma2
A joint 100(1 - α) % confidence region for the elements of
\boldsymbol{\beta}
(\boldsymbol{\hat{\beta}}-b)\operatorname{T}X\operatorname{T}X(\boldsymbol{\hat{\beta}}-b)\leps2F1(p,\nu),
where the variable b represents any point in the confidence region, p is the number of parameters, i.e. number of elements of the vector
\boldsymbol{\beta},
\boldsymbol{\hat{\beta}}
\sigma2
| ||||
| s |
-p}.
Further, F is the quantile function of the F-distribution, with p and
\nu=n-p
\alpha
X\operatorname{T}
X
The expression can be rewritten as:
(\boldsymbol{\hat{\beta}}-b)\operatorname{T}
| -1 | |
| C | |
| \beta |
(\boldsymbol{\hat{\beta}}-b)\lepF1(p,\nu),
where
C\beta=s2\left(X\operatorname{T}X\right)-1
\boldsymbol{\hat{\beta}}
The above inequality defines an ellipsoidal region in the p-dimensional Cartesian parameter space Rp. The centre of the ellipsoid is at the estimate
\boldsymbol{\hat{\beta}}
Now consider the more general case where some distinct elements of
\boldsymbol{\varepsilon}
\boldsymbol{\varepsilon}
V\sigma2
I
It is possible to find[2] a nonsingular symmetric matrix P such that
P\primeP=PP=V
In effect, P is a square root of the covariance matrix V.
The least-squares problem
Y=X\boldsymbol{\beta}+\boldsymbol{\varepsilon}
can then be transformed by left-multiplying each term by the inverse of P, forming the new problem formulation
Z=Q\boldsymbol{\beta}+f,
where
Z=P-1Y
Q=P-1X
f=P-1\boldsymbol{\varepsilon}
A joint confidence region for the parameters, i.e. for the elements of
\boldsymbol{\beta}
(b-\boldsymbol{\hat{\beta}})\primeQ\primeQ(b-\boldsymbol{\hat{\beta}})={
| p | |
| n-p |
Here F represents the percentage point of the F-distribution and the quantities p and n-p are the degrees of freedom which are the parameters of this distribution.
Confidence regions can be defined for any probability distribution. The experimenter can choose the significance level and the shape of the region, and then the size of the region is determined by the probability distribution. A natural choice is to use as a boundary a set of points with constant
\chi2
One approach is to use a linear approximation to the nonlinear model, which may be a close approximation in the vicinity of the solution, and then apply the analysis for a linear problem to find an approximate confidence region. This may be a reasonable approach if the confidence region is not very large and the second derivatives of the model are also not very large.
Bootstrapping approaches can also be used.[4]