Rate function explained

In mathematics - specifically, in large deviations theory - a rate function is a function used to quantify the probabilities of rare events. Such functions are used to formulate large deviation principles. A large deviation principle quantifies the asymptotic probability of rare events for a sequence of probabilities.

A rate function is also called a Cramér function, after the Swedish probabilist Harald Cramér.

Definitions

Rate function An extended real-valued function

I:X\to[0,+infty]

defined on a Hausdorff topological space

X

is said to be a rate function if it is not identically

+infty

and is lower semi-continuous i.e. all the sub-level sets

\{x\inX\midI(x)\leqc\}forc\geq0

are closed in

X

. If, furthermore, they are compact, then

I

is said to be a good rate function.

A family of probability measures

(\mu\delta)\delta

on

X

is said to satisfy the large deviation principle with rate function

I:X\to[0,+infty)

(and rate

1/\delta

) if, for every closed set

F\subseteqX

and every open set

G\subseteqX

,

\limsup\delta\deltalog\mu\delta(F)\leq-infxI(x),(U)

\liminf\delta\deltalog\mu\delta(G)\geq-infxI(x).(L)

If the upper bound (U) holds only for compact (instead of closed) sets

F

, then

(\mu\delta)\delta

is said to satisfy the weak large deviations principle (with rate

1/\delta

and weak rate function

I

).

Remarks

The role of the open and closed sets in the large deviation principle is similar to their role in the weak convergence of probability measures: recall that

(\mu\delta)\delta

is said to converge weakly to

\mu

if, for every closed set

F\subseteqX

and every open set

G\subseteqX

,

\limsup\delta\mu\delta(F)\leq\mu(F),

\liminf\delta\mu\delta(G)\geq\mu(G).

There is some variation in the nomenclature used in the literature: for example, den Hollander (2000) uses simply "rate function" where this article - following Dembo & Zeitouni (1998) - uses "good rate function", and "weak rate function". Rassoul-Agha & Seppäläinen (2015) uses the term "tight rate function" instead of "good rate function" due to the connection with exponential tightness of a family of measures. Regardless of the nomenclature used for rate functions, examination of whether the upper bound inequality (U) is supposed to hold for closed or compact sets tells one whether the large deviation principle in use is strong or weak.

Properties

Uniqueness

A natural question to ask, given the somewhat abstract setting of the general framework above, is whether the rate function is unique. This turns out to be the case: given a sequence of probability measures (μδ)δ>0 on X satisfying the large deviation principle for two rate functions I and J, it follows that I(x) = J(x) for all x ∈ X.

Exponential tightness

It is possible to convert a weak large deviation principle into a strong one if the measures converge sufficiently quickly. If the upper bound holds for compact sets F and the sequence of measures (μδ)δ>0 is exponentially tight, then the upper bound also holds for closed sets F. In other words, exponential tightness enables one to convert a weak large deviation principle into a strong one.

Continuity

Naïvely, one might try to replace the two inequalities (U) and (L) by the single requirement that, for all Borel sets S ⊆ X,

\lim\delta\deltalog\mu\delta(S)=-infxI(x).(E)

The equality (E) is far too restrictive, since many interesting examples satisfy (U) and (L) but not (E). For example, the measure μδ might be non-atomic for all δ, so the equality (E) could hold for S =  only if I were identically +∞, which is not permitted in the definition. However, the inequalities (U) and (L) do imply the equality (E) for so-called I-continuous sets S ⊆ X, those for which

I(\stackrel{\circ}{S})=I(\bar{S}),

where

\stackrel{\circ}{S}

and

\bar{S}

denote the interior and closure of S in X respectively. In many examples, many sets/events of interest are I-continuous. For example, if I is a continuous function, then all sets S such that

S\subseteq\bar{\stackrel{\circ}{S}}

are I-continuous; all open sets, for example, satisfy this containment.

Transformation of large deviation principles

Given a large deviation principle on one space, it is often of interest to be able to construct a large deviation principle on another space. There are several results in this area:

History and basic development

The notion of a rate function emerged in the 1930s with the Swedish mathematician Harald Cramér's study of a sequence of i.i.d. random variables (Zi)i∈

N

. Namely, among some considerations of scaling, Cramér studied the behavior of the distribution of the average X_n = \frac 1 n \sum_^n Z_i as n→∞.[1] He found that the tails of the distribution of Xn decay exponentially as e(x) where the factor λ(x) in the exponent is the Legendre–Fenchel transform (a.k.a. the convex conjugate) of the cumulant-generating function

\PsiZ(t)=log\operatornameEetZ.

For this reason this particular function λ(x) is sometimes called the Cramér function. The rate function defined above in this article is a broad generalization of this notion of Cramér's, defined more abstractly on a probability space, rather than the state space of a random variable.

See also

References

  1. Cramér. Harald. Sur un nouveau théorème-limite de la théorie des probabilités. Colloque consacré à la théorie des probabilités, Part 3, Actualités scientifiques et industrielles. 731. 1938. 5–23. fr.