## Stochastic Processes

A stochastic or random process as opposed to a deterministic process, includes the possibility that a system may evolve in different ways in a way that can only be predicted with probability. For example a differential equation can be solved and the solutions determine the future state of the system but in a stochastic or random process there is some indeterminacy in its future evolution described by probability distributions. This means that even if the initial condition (or starting point) is known, there are many possibilities the process might go to, but some paths may be more probable and others less so.

Deterministic dynamical processes are typically formulated as a set of rules which allow  for the state of the system at time to be found from the state of the system at time t. For stochastic systems, we can only specify the probability of  finding the system in a given state. If this only depends on the state of the system at the  previous time step, but not those before this, the stochastic process is said to be Markovian.  Many stochastic processes are Markovian to a very good approximation.

The mathematical definition of a Markov process follows from the definition of the hierarchy of pdfs for a given process. This involves the joint pdf which is the probability that the system is in state at time state at time and state at time and also the conditional pdf which is the probability that the system is in state at time at time given that it was in state at time at time These pdfs are all non-negative  and normalisable, and relations exist between them due to symmetry and reduction (integration over some of the state variables). For a Markov process the history of the system, apart from the immediate past, is forgotten, and so A direct consequence of this is that the whole hierarchy of pdfs can be determined from  only two of them: and The hierarchy of defining equations then  collapses to only two: (1)

and for (2)

The pdf is referred to as the transition probability and (2) as the Chapman  - Kolmogorov equation. While the pdfs for a Markov process must obey (1) and (2),  the converse also holds: any two non-negative functions and which satisfy  (1) and (2), uniquely define a Markov process. 