Let X0, X1, X2,... be an irreducible Markov chain with state space {1, 2,...,M M 3

Chapter 11, Problem 2

(choose chapter or problem)

Let X0, X1, X2,... be an irreducible Markov chain with state space {1, 2,...,M M 3, transition matrix Q = (qij ), and stationary distribution s = (s1,...,sM). Let the initial state X0 follow the stationary distribution, i.e., P(X0 = i) = si. (a) On average, how many of X0, X1,...,X9 equal 3? (In terms of s; simplify.) (b) Let Yn = (Xn 1)(Xn 2). For M = 3, find an example of Q (the transition matrix for the original chain X0, X1,...) where Y0, Y1,... is Markov, and another example of Q where Y0, Y1,... is not Markov. In your examples, make qii > 0 for at least one i and make sure it is possible to get from any state to any other state eventually.