### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Markov Processes, Queues, and Monte Carlo Simulations APPM 4560

GPA 3.76

### View Full Document

## 50

## 0

## Popular in Course

## Popular in Applied Math

This 4 page Class Notes was uploaded by Dr. Filomena Hegmann on Thursday October 29, 2015. The Class Notes belongs to APPM 4560 at University of Colorado at Boulder taught by Staff in Fall. Since its upload, it has received 50 views. For similar materials see /class/231880/appm-4560-university-of-colorado-at-boulder in Applied Math at University of Colorado at Boulder.

## Reviews for Markov Processes, Queues, and Monte Carlo Simulations

### What is Karma?

#### Karma is the currency of StudySoup.

Date Created: 10/29/15

H 10 9 r gt o1 APPM 45560 Markov Chains Fall 2007 Some Review Problems for Exam One Note This set of problems is longer than the actual exam A Markov chain Xn on the state space 01 has transition probability matrix 0 1 P1 0 08 02 1 05 05 a If initially we are equally likely to start in either state nd PX1 O b Find PX3 1X0 0 c What is the long run proportion of visits the chain will make to state 0 Find all the communication classes of the Markov chain with the given transition probability matrix and classify each class as either recurrent or transient Is the chain irreducible 01234 011100 3 P7101200 231000 3001 0 400001 An urn contains 2 red balls and 1 yellow ball At each point in time a ball is selected at random from the urn and replaced with a yellow ball The selection process continues until all of the red balls are removed from the urn What is the expected duration of the process A rat with a bad cold so he can t smell is put into compartment 4 of the maze shown in Figure 1 At each time step the rat moves to another compartment He never stays where he is He chooses a departure door from each compartment at random We want to know the probability that he nds the food before he gets shocked a Set up an appropriate transition matrix and the system of equations you would need to solve this Do not solve the system b Is this system periodic Aperiodic Neither Consider the Markov chain on X 012 running according to the transition probability matrix H Mm OW H O bli Nh mh t H OMHMH m mule 1 Ken Meze 1m Pzahlem 4 e When 15 nhe lung um pelwmage 0f mme nhen nhe chem 15 m enene 17 b Lf nhe chem snelne m snene 0 when 15 nhe expeened numhel 0f snepe unnl nhe chem hnne snene 17 e Lfche chem 5131 m snene o andxfT 15 nhe msn xme nhen 1n hxcyexchez One afnhe enenee 1 a1 2 when 15 nhe plabehxhcy nhen nhe chem w l he m snene 2 en mme T7 d Scenmg m enene 0 when 15 nhe meeh xme nhen nhe pmeeee spehde m snene 1 pan on msn hxcmng snen 27 e Scenmg m enene 0 when 15 nhe meeh xme nhen nhe pmeeee spehde m snene 1 pan on letmnmg na snene 07 1 End PltX2 11x2 e 2X e 2Xa o chhauc campucmg 1 d0 yau expect PltX2 1 X e 2X0 0 na he smaller a1 lelgel39 Explem va2 en h a Suppaee nhen nhe plabehxlxcy 1n 1eme today 15 03 If nemhel 0f nhe lesn twa day wee lemy hun 061i en leesn One 0f nhe lesn twa day wee lemy e Sen nhzepmhlem up ee afaul snene Mezkav chem b When 15 nhe plabehxhcy nhen 1n w l 1em ah Wednesday gven nhen n ma han 1em ah Sunday a1 Mandey e Bud nhe snemahesy dxmxbucxan m the chem Len 7 he e stenanely dxsrnbuman 0f eMezkav chem end ten 1 and 7 he twa snenee such nhen gt0 andz y Shawchez j gt0 Below is an actual old exam APPM 45560 Markov Chains Fall 2004 Exam One In Class Part Answer any four out of the following siix questions If you attempt more than four clearly specify the four you want graded or your rst four problems will be graded 1 A Markov chain Xn on the state space 12 has transition probability matrix 1 2 P 1 06 04 2 03 07 a If initially we are equally likely to start in either state nd PX2 2 b What is the long run proportion of visits the chain will make to state 2 2 An urn contains two red bugs and one green bug Bugs are chosen at random one by one from the urn If a red bug is chosen it is removed If the green bug is chosen it is returned to the urn The selection process continues until all of the red bugs are removed from the urn What is the mean duration of the game 3 Consider the Markov chain whose transition probability matrix is given by 0 1 2 0 1 0 0 P 7 1 02 07 01 2 0 0 1 SET UP BUT DO NOT SOLVE systems of equations to answer the following two questions De ne variables used and specify what needs to be solved for a Starting in state 1 determine the probability that the Markov chain ends in state 0 b Determine the mean time to absorption for this Markov chain 4 A Markov chain on states 0 123 4 5 has a transition probability matrix given by 0 1 2 3 4 5 1 0 0 0 0 0 23 13 0 0 0 18 78 0 0 16 12 0 16 16 14 0 13 524 524 0 0 0 0 0 01HgtODMHO HOOOOO Find all communication classes and specify whether the states in each are transient or recur rent U1 9 Consider the Markov chain on S 012 running according to the transition probability matrix 012 01 P 3H 12 2110 Specify how to nd PX5 2X4 7amp1X3 7amp1X2 7amp1X1 7amp1X0 0 You need not actually compute the answer For the Markov chain given in problem 5 above SET UP BUT DO NOT SOLVE systems of equations to answer the following questions a Find the stationary distribution b If the chain starts in state 2 nd the expected number of times the chain hits state 1 before hitting state 0

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.