Pseudolikelihood Explained

In statistical theory, a pseudolikelihood is an approximation to the joint probability distribution of a collection of random variables. The practical use of this is that it can provide an approximation to the likelihood function of a set of observed data which may either provide a computationally simpler problem for estimation, or may provide a way of obtaining explicit estimates of model parameters.

The pseudolikelihood approach was introduced by Julian Besag in the context of analysing data having spatial dependence.

Definition

Given a set of random variables

X = X _{1,} X _{2,} \ldots, X _{n} the pseudolikelihood of X = x = (x _{1,x} _{2,} \ldots, x _{n)} isL(\theta) : = \prod _{i} Pr _{\theta(X} _{i} = x _{i\mid} X _{j} = x _{j} for j ≠ i) = \prod _{i} Pr _{\theta} (X _{i} = x _{i} \mid X _{-i} = x _{-i} )

in discrete case and

L(\theta) : = \prod _{i} p _{\theta(x} _{i} \mid x _{j} for j ≠ i) = \prod _{i} p _{\theta} (x _{i} \mid x _{-i} ) = \prod _{i} p _{\theta} (x _{i} \mid x _{1,\ldots,} \hat x _{i,} \ldots, x _{n)}

in continuous one. Here

X is a vector of variables, x is a vector of values, p _{\theta( ⋅} \mid ⋅ ) is conditional density and \theta = (\theta _{1,} \ldots, \theta _{p)} is the vector of parameters we are to estimate. The expression X = x above means that each variable X _{i} in the vector X has a corresponding value x _{i} in the vector x and x _{-i} = (x _{1,} \ldots,\hat x _{i,} \ldots, x _{n)} means that the coordinate x _{i} has been omitted. The expression Pr _{\theta(X} = x) is the probability that the vector of variables X has values equal to the vector x . This probability of course depends on the unknown parameter \theta . Because situations can often be described using state variables ranging over a set of possible values, the expression Pr _{\theta(X} = x) can therefore represent the probability of a certain state among all possible states allowed by the state variables.

The pseudo-log-likelihood is a similar measure derived from the above expression, namely (in discrete case)

l(\theta): =log L(\theta) = \sum _{i} log Pr _{\theta(X} _{i} = x _{i\mid} X _{j} = x _{j} for j ≠ i).

One use of the pseudolikelihood measure is as an approximation for inference about a Markov or Bayesian network, as the pseudolikelihood of an assignment to

X _{i} may often be computed more efficiently than the likelihood, particularly when the latter may require marginalization over a large number of variables.

Properties

Use of the pseudolikelihood in place of the true likelihood function in a maximum likelihood analysis can lead to good estimates, but a straightforward application of the usual likelihood techniques to derive information about estimation uncertainty, or for significance testing, would in general be incorrect.^[1]

Notes and References

1. Dodge, Y. (2003) The Oxford Dictionary of Statistical Terms, Oxford University Press.