A stochastic or random process as opposed to a deterministic process, includes the possibility that a system may evolve in different ways in a way that can only be predicted with probability. For example a differential equation can be solved and the solutions determine the future state of the system but in a stochastic or random process there is some indeterminacy in its future evolution described by probability distributions. This means that even if the initial condition (or starting point) is known, there are many possibilities the process might go to, but some paths may be more probable and others less so.

Deterministic dynamical processes are typically formulated as a set of rules which allow for the state of the system at timeto be found from the state of the system at time t. For stochastic systems, we can only specify the probability of finding the system in a given state. If this only depends on the state of the system at the previous time step, but not those before this, the stochastic process is said to be Markovian. Many stochastic processes are Markovian to a very good approximation.

The mathematical definition of a Markov process follows from the definition of the hierarchy of pdfs for a given process. This involves the joint pdfwhich is the probability that the system is in stateat timestateat timeand state at timeand also the conditional pdfwhich is the probability that the system is in stateat timeat timegiven that it was in state at timeat timeThese pdfs are all non-negative and normalisable, and relations exist between them due to symmetry and reduction (integration over some of the state variables). For a Markov process the history of the system, apart from the immediate past, is forgotten, and so

A direct consequence of this is that the whole hierarchy of pdfs can be determined from only two of them:andThe hierarchy of defining equations then collapses to only two:

(1)

and

for(2)

The pdfis referred to as the transition probability and (2) as the Chapman - Kolmogorov equation. While the pdfs for a Markov process must obey (1) and (2), the converse also holds: any two non-negative functionsandwhich satisfy (1) and (2), uniquely define a Markov process.