In statistics, adaptive or "variable-bandwidth" kernel density estimation is a form of kernel density estimation in which the size of the kernels used in the estimate are varieddepending upon either the location of the samples or the location of the test point.It is a particularly effective technique when the sample space is multi-dimensional.[1]
Given a set of samples,
\lbrace\vecxi\rbrace
P(\vecx)
\vecx
P(\vecx) ≈
| W | |
| nhD |
W=
| n | |
| \sum | |
| i=1 |
wi
wi=K\left(
| \vecx-\vecxi | |
| h |
\right)
where n is the number of samples, K is the "kernel", h is its width and D is the number of dimensions in
\vecx
Using a fixed filter width may mean that in regions of low density, all sampleswill fall in the tails of the filter with very low weighting, while regions of highdensity will find an excessive number of samples in the central region with weightingclose to unity. To fix this problem, we vary the width of the kernel in differentregions of the sample space. There are two methods of doing this: balloon and pointwise estimation.In a balloon estimator, the kernel width is varied depending on the location of the test point. In a pointwise estimator, the kernel width is varied dependingon the location of the sample.[1]
For multivariate estimators, the parameter, h, can be generalized tovary not just the size, but also the shape of the kernel. This more complicated approachwill not be covered here.
A common method of varying the kernel width is to make it inversely proportional to the density at the test point:
h=
| k | |
| \left[nP(\vecx)\right]1/D |
where k is a constant. If we back-substitute the estimated PDF, and assuming a Gaussian kernel function,we can show that W is a constant:
W=kD(2\pi)D/2
A similar derivation holds for any kernel whose normalising function is of the order, although with a different constant factor in place of the term. This produces a generalization of the k-nearest neighbour algorithm.That is, a uniform kernel function will return the KNN technique.[2]
There are two components to the error: a variance term and a bias term. The variance term is given as:[1]
e1=
| P\intK2 | |
| nhD |
The bias term is found by evaluating the approximated function in the limit as the kernelwidth becomes much larger than the sample spacing. By using a Taylor expansion for the real function, the bias term drops out:
e2=
| h2 | |
| n |
\nabla2P
An optimal kernel width that minimizes the error of each estimate can thus be derived.
The method is particularly effective when applied to statistical classification.There are two ways we can proceed: the first is to compute the PDFs ofeach class separately, using different bandwidth parameters, and then compare them as in Taylor.[3] Alternatively, we can divide up the sum based on the class of each sample:
P(j,\vec{x}) ≈
| 1 | |
| n |
| n | |
| \sum | |
| i=1,ci=j |
wi
where ci is the class of the ith sample.The class of the test point may be estimated through maximum likelihood.