×

### Let's log you in.

or

Don't have a StudySoup account? Create one here!

×

or

## Markov Processes, Queues, and Monte Carlo Simulations

by: Dr. Filomena Hegmann

50

0

4

# Markov Processes, Queues, and Monte Carlo Simulations APPM 4560

Marketplace > University of Colorado at Boulder > Applied Math > APPM 4560 > Markov Processes Queues and Monte Carlo Simulations
Dr. Filomena Hegmann

GPA 3.76

Staff

These notes were just uploaded, and will be ready to view shortly.

Either way, we'll remind you when they're ready :)

Get a free preview of these Notes, just enter your email below.

×
Unlock Preview

COURSE
PROF.
Staff
TYPE
Class Notes
PAGES
4
WORDS
KARMA
25 ?

## Popular in Applied Math

This 4 page Class Notes was uploaded by Dr. Filomena Hegmann on Thursday October 29, 2015. The Class Notes belongs to APPM 4560 at University of Colorado at Boulder taught by Staff in Fall. Since its upload, it has received 50 views. For similar materials see /class/231880/appm-4560-university-of-colorado-at-boulder in Applied Math at University of Colorado at Boulder.

×

## Reviews for Markov Processes, Queues, and Monte Carlo Simulations

×

×

### What is Karma?

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/29/15
H 10 9 r gt o1 APPM 45560 Markov Chains Fall 2007 Some Review Problems for Exam One Note This set of problems is longer than the actual exam A Markov chain Xn on the state space 01 has transition probability matrix 0 1 P1 0 08 02 1 05 05 a If initially we are equally likely to start in either state nd PX1 O b Find PX3 1X0 0 c What is the long run proportion of visits the chain will make to state 0 Find all the communication classes of the Markov chain with the given transition probability matrix and classify each class as either recurrent or transient Is the chain irreducible 01234 011100 3 P7101200 231000 3001 0 400001 An urn contains 2 red balls and 1 yellow ball At each point in time a ball is selected at random from the urn and replaced with a yellow ball The selection process continues until all of the red balls are removed from the urn What is the expected duration of the process A rat with a bad cold so he can t smell is put into compartment 4 of the maze shown in Figure 1 At each time step the rat moves to another compartment He never stays where he is He chooses a departure door from each compartment at random We want to know the probability that he nds the food before he gets shocked a Set up an appropriate transition matrix and the system of equations you would need to solve this Do not solve the system b Is this system periodic Aperiodic Neither Consider the Markov chain on X 012 running according to the transition probability matrix H Mm OW H O bli Nh mh t H OMHMH m mule 1 Ken Meze 1m Pzahlem 4 e When 15 nhe lung um pelwmage 0f mme nhen nhe chem 15 m enene 17 b Lf nhe chem snelne m snene 0 when 15 nhe expeened numhel 0f snepe unnl nhe chem hnne snene 17 e Lfche chem 5131 m snene o andxfT 15 nhe msn xme nhen 1n hxcyexchez One afnhe enenee 1 a1 2 when 15 nhe plabehxhcy nhen nhe chem w l he m snene 2 en mme T7 d Scenmg m enene 0 when 15 nhe meeh xme nhen nhe pmeeee spehde m snene 1 pan on msn hxcmng snen 27 e Scenmg m enene 0 when 15 nhe meeh xme nhen nhe pmeeee spehde m snene 1 pan on letmnmg na snene 07 1 End PltX2 11x2 e 2X e 2Xa o chhauc campucmg 1 d0 yau expect PltX2 1 X e 2X0 0 na he smaller a1 lelgel39 Explem va2 en h a Suppaee nhen nhe plabehxlxcy 1n 1eme today 15 03 If nemhel 0f nhe lesn twa day wee lemy hun 061i en leesn One 0f nhe lesn twa day wee lemy e Sen nhzepmhlem up ee afaul snene Mezkav chem b When 15 nhe plabehxhcy nhen 1n w l 1em ah Wednesday gven nhen n ma han 1em ah Sunday a1 Mandey e Bud nhe snemahesy dxmxbucxan m the chem Len 7 he e stenanely dxsrnbuman 0f eMezkav chem end ten 1 and 7 he twa snenee such nhen gt0 andz y Shawchez j gt0 Below is an actual old exam APPM 45560 Markov Chains Fall 2004 Exam One In Class Part Answer any four out of the following siix questions If you attempt more than four clearly specify the four you want graded or your rst four problems will be graded 1 A Markov chain Xn on the state space 12 has transition probability matrix 1 2 P 1 06 04 2 03 07 a If initially we are equally likely to start in either state nd PX2 2 b What is the long run proportion of visits the chain will make to state 2 2 An urn contains two red bugs and one green bug Bugs are chosen at random one by one from the urn If a red bug is chosen it is removed If the green bug is chosen it is returned to the urn The selection process continues until all of the red bugs are removed from the urn What is the mean duration of the game 3 Consider the Markov chain whose transition probability matrix is given by 0 1 2 0 1 0 0 P 7 1 02 07 01 2 0 0 1 SET UP BUT DO NOT SOLVE systems of equations to answer the following two questions De ne variables used and specify what needs to be solved for a Starting in state 1 determine the probability that the Markov chain ends in state 0 b Determine the mean time to absorption for this Markov chain 4 A Markov chain on states 0 123 4 5 has a transition probability matrix given by 0 1 2 3 4 5 1 0 0 0 0 0 23 13 0 0 0 18 78 0 0 16 12 0 16 16 14 0 13 524 524 0 0 0 0 0 01HgtODMHO HOOOOO Find all communication classes and specify whether the states in each are transient or recur rent U1 9 Consider the Markov chain on S 012 running according to the transition probability matrix 012 01 P 3H 12 2110 Specify how to nd PX5 2X4 7amp1X3 7amp1X2 7amp1X1 7amp1X0 0 You need not actually compute the answer For the Markov chain given in problem 5 above SET UP BUT DO NOT SOLVE systems of equations to answer the following questions a Find the stationary distribution b If the chain starts in state 2 nd the expected number of times the chain hits state 1 before hitting state 0

×

×

### BOOM! Enjoy Your Free Notes!

×

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

Jim McGreen Ohio University

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Kyle Maynard Purdue

#### "When you're taking detailed notes and trying to help everyone else out in the class, it really helps you learn and understand the material...plus I made \$280 on my first study guide!"

Steve Martinelli UC Los Angeles

Forbes

#### "Their 'Elite Notetakers' are making over \$1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!
×

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com