Markov additive process explained

In applied probability, a Markov additive process (MAP) is a bivariate Markov process where the future states depends only on one of the variables.

Definition

Finite or countable state space for J(t)

The process

\{ (X(t), J(t)) : t\ge 0 \} is a Markov additive process with continuous time parameter t if^[1] \{ (X(t), J(t)) ; t\ge 0 \} is a Markov process

1. the conditional distribution of

(X(t + s) - X(t), J(t + s)) given (X(t), J(t)) depends only on J(t) .

The state space of the process is R × S where X(t) takes real values and J(t) takes values in some countable set S.

General state space for J(t)

For the case where J(t) takes a more general state space the evolution of X(t) is governed by J(t) in the sense that for any f and g we require^[2]

E[f(X _{t+s} - X _{t)g(J} _{t+s} )| l F _{t]} = E _{J _{t,0}} [f(X _{s)g(J} _{s)]} .

Example

A fluid queue is a Markov additive process where J(t) is a continuous-time Markov chain.

Applications

Çinlar uses the unique structure of the MAP to prove that, given a gamma process with a shape parameter that is a function of Brownian motion, the resulting lifetime is distributed according to the Weibull distribution.

Kharoufeh presents a compact transform expression for the failure distribution for wear processes of a component degrading according to a Markovian environment inducing state-dependent continuous linear wear by using the properties of a MAP and assuming the wear process to be temporally homogeneous and that the environmental process has a finite state space.

Notes and References

1. Book: Magiera . R. . Optimal Sequential Estimation for Markov-Additive Processes . 10.1007/978-1-4612-2234-7_12 . Advances in Stochastic Models for Reliability, Quality and Safety . 167–181 . 1998 . 978-1-4612-7466-7 .
2. Book: S. R. . Asmussen. 10.1007/0-387-21525-5_11 . Markov Additive Models . Applied Probability and Queues . Stochastic Modelling and Applied Probability . 51 . 302–339 . 2003 . 978-0-387-00211-8 .