Markov Chains – Now to Next

In the last post, we started an example to demonstrate the objective of Markov processes.

The question is, what is the expected number of customers in shops A, B, and C in the following week?

The whole formulation is nothing but future state = present state x a probability matrix. The probability matrix, fundamental to making this translation, is developed from the diagram we created earlier.

The first row of the matrix are the probabilities: A to A, B to A and C to A
The second row: A to B, B to B and C to B
The third row: A to C, B to C and C to C

Perform the matrix multiplication between the two. Here are the R codes.

pM <- matrix(c(0.8, 0.1, 0.1, 0.2, 0.7, 0.1, 0.1, 0.3, 0.6), nrow = 3)
pM

    [,1] [,2] [,3]
[1,]  0.8  0.2  0.1
[2,]  0.1  0.7  0.3
[3,]  0.1  0.1  0.6

pM <- matrix(c(0.8, 0.1, 0.1, 0.2, 0.7, 0.1, 0.1, 0.3, 0.6), nrow = 3)
past <- c(0.4, 0.24, 0.36)
future <- pM%*%past
future

      [,1]
[1,] 0.404
[2,] 0.316
[3,] 0.280

What are Markov Chains: An Introduction: Michel van Biezen