Expected long-term frequency interpretation. Consider a Markov chain with a single recurrent class which is aperiodic. Show that 1. Vij (n) 7rj = 1m --, n-oo n for all i, j = 1, . . . , m, where the 7rj are the steady-state probabilities, and Vij(n) is the expected value of the number of visits to state j within the first n transitions, starting from state i. Hint: Use the following fact from analysis. If a sequence an converges to a number a, the sequence bn defined by bn = (lin) 2:=1 ak also converges to a. Solution. We first assert that for all n, i, and j, we have n Vij(n) = L rij(k). k=l To see this, note that where h is the random variable that takes the value 1 if X k = j, and the value 0 otherwise, so that Since E [h I Xo = i] = rij(k). n Vij(n) = .!. '"' rij(k), n n L k=l and rij(k) converges to 7rj, it follows that vij(n)ln also converges to 7rj, which is the desired result. For completeness, we also provide the proof of the fact given in the hint, and which was used in the last step of the above argument. Consider a sequence an that converges to some a, and let bn = (lin) 2:=1 ak. Fix some f > O. Since an converges to a, there exists some no such that ak a+(f/2), for all k > no. Let also c = maxk ak. We then have The limit of the right-hand side, as n tends to infinity, is a + (f/2). Therefore, there exists some nl such that bn a + f, for every n 2: nl . By a symmetrical argument, there exists some n2 such that bn a - , for every n n2. We have shown that for every > 0, there exists some n3 (namely, n3 = max{nl , n2}) such that Ibn - a l ::; , for all n n3. This means that bn converges to a.

ST 512 Week Five Notes MaLyn Lawhorn September 11,2017 and September 13, 2017 Correlation and Association It is important to determine whether or not there are relationships among random variables X and Y . Some questions and procedures to keep in mind when determining this are 1. Is there a linear association between X and Y 2. If a linear association exists, how strong is it 3. To what degree/extent is y 2 Y explained by x 2 X 4. We use question 3 to estimate the average y for a given x. 5. We use question 3 to predict future y for a given x. Note that points 4 and 5 above are evaluating different things. Point 4 is focusing on the average y whereas the y in point 5 will be a speciﬁc yield. Addi