Markov Process

MARKOV PROCESS

Definition: Markov Process [A.U A/M 2015 (RP) R-8]

A random process X(t) is said to be Markov process, if

(OR)

"If the future value depends only on the present state but not on the past states is called a Markov process".

Note: X₁, X₂, ..., X_n-1, X_n are known as the states of the process, by our definition "future state X_n+1" depends only on "present state X_n", but not on "past states X₁, X₂, X₃, ..., X_n-1

Example: Markov Process

1. The probability of raining today depends only on previous weather conditions existed for the last two days and not on past weather conditions.

2. A first-order linear differential or difference equation is Markovian.

3. Weather prediction models

Classification of Markov Process

A Markov process can be classified into four types, based on the values taken by time t and the state space {X_i}.

(i) A continuous random process satisfying Markov property is known as continuous parameter Markov Process. Note that for a continuous random process both t and {X_i} are continuous.

(ii) A continuous random sequence satisfying Markov property is known as discrete parameter Markov process as the parameter t is discrete but {X_i} is continuous.

(iii) A discrete random sequence satisfying Markov property is known as the discrete parameter Markov Chain as t is discrete and {X_i} is also discrete.

(iv) A discrete random process satisfying Markov property is called as continuous parameter Markov chain because the parameter t is continuous and {X_i} is discrete.