×
Log in to StudySoup
Get Full Access to Introduction To Probability, - 2 Edition - Chapter 7 - Problem 21
Join StudySoup for FREE
Get Full Access to Introduction To Probability, - 2 Edition - Chapter 7 - Problem 21

Already have an account? Login here
×
Reset your password

Expected long-term frequency interpretation. Consider a

Introduction to Probability, | 2nd Edition | ISBN: 9781886529236 | Authors: Dimitri P. Bertsekas John N. Tsitsiklis ISBN: 9781886529236 227

Solution for problem 21 Chapter 7

Introduction to Probability, | 2nd Edition

  • Textbook Solutions
  • 2901 Step-by-step solutions solved by professors and subject experts
  • Get 24/7 help from StudySoup virtual teaching assistants
Introduction to Probability, | 2nd Edition | ISBN: 9781886529236 | Authors: Dimitri P. Bertsekas John N. Tsitsiklis

Introduction to Probability, | 2nd Edition

4 5 1 379 Reviews
13
1
Problem 21

Expected long-term frequency interpretation. Consider a Markov chain with a single recurrent class which is aperiodic. Show that 1. Vij (n) 7rj = 1m --, n-oo n for all i, j = 1, . . . , m, where the 7rj are the steady-state probabilities, and Vij(n) is the expected value of the number of visits to state j within the first n transitions, starting from state i. Hint: Use the following fact from analysis. If a sequence an converges to a number a, the sequence bn defined by bn = (lin) 2:=1 ak also converges to a. Solution. We first assert that for all n, i, and j, we have n Vij(n) = L rij(k). k=l To see this, note that where h is the random variable that takes the value 1 if X k = j, and the value 0 otherwise, so that Since E [h I Xo = i] = rij(k). n Vij(n) = .!. '"' rij(k), n n L k=l and rij(k) converges to 7rj, it follows that vij(n)ln also converges to 7rj, which is the desired result. For completeness, we also provide the proof of the fact given in the hint, and which was used in the last step of the above argument. Consider a sequence an that converges to some a, and let bn = (lin) 2:=1 ak. Fix some f > O. Since an converges to a, there exists some no such that ak a+(f/2), for all k > no. Let also c = maxk ak. We then have The limit of the right-hand side, as n tends to infinity, is a + (f/2). Therefore, there exists some nl such that bn a + f, for every n 2: nl . By a symmetrical argument, there exists some n2 such that bn a - , for every n n2. We have shown that for every > 0, there exists some n3 (namely, n3 = max{nl , n2}) such that Ibn - a l ::; , for all n n3. This means that bn converges to a.

Step-by-Step Solution:
Step 1 of 3

ST 512 Week Five Notes MaLyn Lawhorn September 11,2017 and September 13, 2017 Correlation and Association It is important to determine whether or not there are relationships among random variables X and Y . Some questions and procedures to keep in mind when determining this are 1. Is there a linear association between X and Y 2. If a linear association exists, how strong is it 3. To what degree/extent is y 2 Y explained by x 2 X 4. We use question 3 to estimate the average y for a given x. 5. We use question 3 to predict future y for a given x. Note that points 4 and 5 above are evaluating different things. Point 4 is focusing on the average y whereas the y in point 5 will be a specific yield. Addi

Step 2 of 3

Chapter 7, Problem 21 is Solved
Step 3 of 3

Textbook: Introduction to Probability,
Edition: 2
Author: Dimitri P. Bertsekas John N. Tsitsiklis
ISBN: 9781886529236

The answer to “Expected long-term frequency interpretation. Consider a Markov chain with a single recurrent class which is aperiodic. Show that 1. Vij (n) 7rj = 1m --, n-oo n for all i, j = 1, . . . , m, where the 7rj are the steady-state probabilities, and Vij(n) is the expected value of the number of visits to state j within the first n transitions, starting from state i. Hint: Use the following fact from analysis. If a sequence an converges to a number a, the sequence bn defined by bn = (lin) 2:=1 ak also converges to a. Solution. We first assert that for all n, i, and j, we have n Vij(n) = L rij(k). k=l To see this, note that where h is the random variable that takes the value 1 if X k = j, and the value 0 otherwise, so that Since E [h I Xo = i] = rij(k). n Vij(n) = .!. '"' rij(k), n n L k=l and rij(k) converges to 7rj, it follows that vij(n)ln also converges to 7rj, which is the desired result. For completeness, we also provide the proof of the fact given in the hint, and which was used in the last step of the above argument. Consider a sequence an that converges to some a, and let bn = (lin) 2:=1 ak. Fix some f > O. Since an converges to a, there exists some no such that ak a+(f/2), for all k > no. Let also c = maxk ak. We then have The limit of the right-hand side, as n tends to infinity, is a + (f/2). Therefore, there exists some nl such that bn a + f, for every n 2: nl . By a symmetrical argument, there exists some n2 such that bn a - , for every n n2. We have shown that for every > 0, there exists some n3 (namely, n3 = max{nl , n2}) such that Ibn - a l ::; , for all n n3. This means that bn converges to a.” is broken down into a number of easy to follow steps, and 343 words. The full step-by-step solution to problem: 21 from chapter: 7 was answered by , our top Statistics solution expert on 01/09/18, 07:43PM. This textbook survival guide was created for the textbook: Introduction to Probability,, edition: 2. This full solution covers the following key subjects: . This expansive textbook survival guide covers 9 chapters, and 326 solutions. Introduction to Probability, was written by and is associated to the ISBN: 9781886529236. Since the solution to 21 from 7 chapter was answered, more than 234 students have viewed the full step-by-step answer.

Other solutions

People also purchased

Related chapters

Unlock Textbook Solution

Enter your email below to unlock your verified solution to:

Expected long-term frequency interpretation. Consider a