### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Probability MT 426

BC

GPA 3.53

### View Full Document

## 25

## 0

## Popular in Course

## Popular in Mathematics (M)

This 111 page Class Notes was uploaded by Mr. Halie Wilkinson on Saturday October 3, 2015. The Class Notes belongs to MT 426 at Boston College taught by Staff in Fall. Since its upload, it has received 25 views. For similar materials see /class/218065/mt-426-boston-college in Mathematics (M) at Boston College.

## Popular in Mathematics (M)

## Reviews for Probability

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/03/15

MT426 Notebook 1 prepared by Professor Jenny Baglivo Copyright 2009 by Jenny Ar Baglivor All Rights Reserved 1 MT426 Notebook 1 3 1 1 Introduction r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 3 I ll De nitions r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 3 12 Probability Distributions r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 5 1 21 Kolmogorov Axioms r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 5 122 Properties of Kolmogorov Axioms r r r r r r r r r r r r r r r r r r r r r r r r r r 5 123 Long Run Interpretation of Probability r r r r r r r r r r r r r r r r r r r r r r r 8 124 Equally Likely Outcomes r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 9 125 Outcomes With Probabilities Following a Geometric Sequence r r r r r r r r r r 10 1 3 Counting Methods r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 12 1 31 Multiplication Rule Sampling WithWithout Replacement r r r r r r r r r r r r 12 1 32 Permutations Combinations Binomial Coef cients Binomial Theorem r r r r r 15 133 Simple Urn Model Maximum Likelihood Estimation r r r r r r r r r r r r r r r r 17 1 34 Partitioning Sets Multinomial Coef cients Multinomial Theorem r r r r r r r r 20 1 4 Conditional Probability r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 22 1 41 Multiplication Rule For Probability r r r r r r r r r r r r r r r r r r r r r r r r r 23 142 Law of Total Probability Law of Average Conditional Probability r r r r r r r r 25 1 43 Bayes Rule Prior and Posterior Probabilities Diagnostic Tests r r r r r r r r r r 26 15 Independent Events r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 29 1 51 Mutually Independent Events r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 30 1 52 Repeated Trials and Mutual Independence r r r r r r r r r r r r r r r r r r r r r r 31 1 MT426 Notebook 1 This notebook is concerned with introductory probability concepts The notes correspond to material in Chapter 1 of the Rice textbook 11 Introduction Probability is the study of random phenomena Probability theory can be applied for exam ple to study games of chance eg roulette games card games occurrences of catastrophic events e g tornados earthquakes survival of animal species the relationship between genetic variation and disease and changes in stock and commodity markets 111 De nitions 1 The term experiment or random experiment is used in probability theory to describe a procedure whose outcome is not known in advance with certainty Further experiments are assumed to be repeatable at least in theory and to have a well de ned set of possible outcomes 2 The sample space 9 is the set of all possible outcomes of an experiment An event is a subset of the sample space A simple event is an event with a single outcome Events are usually denoted by capital letters A B C and outcomes by lower case letters x y 2 If x E A is observed then A is said to have occurred The favorable outcomes of an experiment form the event of interest 3 Each repetition of an experiment is called a trial Repeated trials are repetitions of the experiment using the speci ed procedure with the outcomes of the trials having no in uence on one another Example 1 You toss a fair coin ve times and record It for head or t for tail each time The sample space is the collection of 32 25 sequences of 5 It s or t s Q filthiii hhhht hhhth hhthh hthhh thhhh hhhtt hhtht hthht thhht hhtth hthth thhth htthh ththh tthhh ttthh tthth thtth httth tthht ththt httht thhtt hthtt hhttt htttt thttt tthtt tttht tttth ttttt If you are interested in getting exactly 5 heads then the event of interest is the simple event A hhhhh If you are interested in getting exactly 3 heads then the event of interest is A hhhtt hhtht hthht thhht hhtth hthth thhth htthh ththh tthhh Footnote 1 let the computer simulate 500 repetitions of the experiment described above The following table gives the number of times 0 1 5 heads were observed NumberofHeads 0 l 1 l 2 l 3 4 l 5 Number of Times 12 l 75 l 159 l 160 l 81 l 13 l In particular exactly 3 heads were observed 32 160500 of the time Example 2 You toss a fair coin until you get a tail and record the sequence of Ms and is The sample space is Q t ht hht hhht hhhht If you are interested in obtaining a tail in three or fewer tosses then the event of interest is A t ht hht Footnote I let the computer simulate 500 repetitions of the experiment described above The following table gives the number of heads in each sequence the number of elements in each sequences and the number of times each type of sequences was observed NumberofHeads Ho 1 2 3 4 5 6 7 8 9 10 11 Number of Elements 1 2 3 4 5 6 7 8 9 10 11 12 Number ofTimas 236 118 70 33 21 10 6 2 2 1 Note that the longest observed sequence had 12 elements In addition sequences of length 3 or less were observed 848 424500 of the time Example 3 Consider a wall that is 10 feet wide and 8 feet tall and a circular da1t board of radius 2 feet located at the center of the wall Assume that each time you throw a dart in the direction of the board the point of impact is somewhere on the w You throw a dart once and record the coordinates of the point of impact using the coordinate system shown on the right below The sample space is the collection of point Qzy 0 z 100 y 8 If you are interested in hitting the dam board then the event of interest is Altzygt z 75gt2y742 4 Footnote Assume further that your aim is so poor that the point of impact is as likely to be in any subregion of the wall as in any other subregion of the same size I let the computer simulate 500 repetitions of the experiment using these assumptions The dart hit the dart board 17 85500 of the time 12 Probability Distributions The basic rules or axioms of probability were introduced by A Kolmogorov in the 1930 s 121 Kolrnogorov Axioms Let A Q Q be an event and PA be the probability that A will occur A probability distribution or simply a probability on a sample space 9 is a speci cation of numbers PA satisfying the following axioms H Full Space Rule PQ 1 to Range Rule If A is an event then 0 PA 1 OJ Disjoint Union Rule lf A1 and A2 are disjoint events that is if A1 A2 0 then PA1 U A2 0i Pairwise Disjoint Union Rule More generally if A1 A2 are pairwise disjoint events that is Ai Aj 0 when i 7 j then PltA1uA2ugtPltA1pA2 If the sequence of events is in nite then the right hand side is understood to be the sum of a convergent in nite series Since 9 is the set of all possible outcomes an outcome in Q is certain to occur the probability of an event that is certain to occur must be 1 axiom 1 Probabilities must be between 0 and 1 axiom 2 and probabilities must be additive when events are pairwise disjoint axiom 3 122 Properties of Kolrnogorov Axioms The following properties can be proven using the Kolmogorov axioms A Complement Rule Let A0 be the complement of A in Q A0 Q 7 A Then PAc 17 PA B Empty Space Rule P O C Subset Rule If A is a subset of B then PA PB D InelusionEwelusion Rule If A and B are events then PA o B PA PB 7 PA m B Exercise Use the Kolmogorov axioms to prove the four properties Exercise Use the axioms and properties to answer each question below a Suppose that PA 03 PB 09 and A U B 9 Compute PA O B PAc PB0 and PAc BC 1 Suppose that PA 03 PB 06 and PA U BC 05 Compute PA O B and PA U B 123 Long Run mayhemieh of Probability The pmhahahcy of an eveac nah be haphaea as Lhe 11mm ufxelauve hequemes Specx cally a a m1 of Lhe expmmenc Lhaa PM hm m h the lung huh the ahsmea empower of hme eveac A is ahsmea ls close e PA Ewwl 1 mnhlwod Canada esshg a fan cum 5 hmes aha hemth h m head a t m caal each Lime and let A Lhe eveac that you Dbsme study a heads m che ve 0536 I Di Elame fzequaqcxas fut Lbs smulahah munn n A Qucxtwn From Lhe aesmphah auhe eapemmehc can you gum Lhe wine of PA7 7 a Yd dam h Lhe ahemah of Lhe beam Lhe pamc ufxmpacb ls somewhere ah Lhe wa11huc Lhac you r Qucxtwn From Lhe aesmphah auhe eapemmehc can you gum Lhe wine of PA7 124 Equally Likely Outcomes Suppose that Q is a nite set with N elements and A C Q is an event with n elements If each outcome is equally likely then the probability of A is 11141 PAWN In this formula 5 represents the number of elements in the nite set S Exercise Eight chips numbered 1 2 8 are placed in an urn a Sample Two Chips With Replacement An experiment consists of thoroughly mixing the A 9 contents of the urn removing one chip and recording the number on the chip replacing the chip to the urn thoroughly mixing the contents again and removing one chip and recording the number on the chip The sample space for the experiment can be represented as follows 2 11 12 13 14 15 16 17 18 21 22 23 24 25 26 27 28 31 32 33 34 35 36 37 38 41 42 43 44 45 46 47 48 51 52 53 54 55 56 57 58 61 62 63 64 65 66 67 68 71 72 73 74 75 76 77 78 81 82 83 84 85 86 87 88 Let A be the event that the sum of the numbers is 6 Then A L and PA Sample Two Chips Without Replacement An experiment consists of thoroughly mixing the contents of the urn removing one chip and recording the number on the chip thor oughly mixing the remaining contents and removing one chip and recording the number on the chip The sample space for the experiment can be represented as follows 2 12 13 14 15 16 17 18 21 23 24 25 26 27 28 31 32 34 35 36 37 38 41 42 43 45 46 47 48 51 52 53 54 56 57 58 61 62 63 64 65 67 68 71 72 73 74 75 76 78 81 82 83 84 85 86 87 Let A be the event that the sum of the numbers is 6 Then A i and PA 125 Outcomes With Probabilities Following a Geometric Sequence Geometric sequences and series are used often in probability A typical set up is as follows 0 Q is a countably in nite set of outcomes7 Q w07w17w27 7wm 7 and o the probabilities of the simple events form a geometric sequence Pwn 1719 297 n 0717 where p is a proportion satisfying 0 lt p lt 1 Exercise Using the setup above7 a Find a general formula for 2310 1 Demonstrate that PQ 220 Pwn 1 Exercise Three chips with the letter a and 5 chips with the letter 1 are placed in an urn The steps thoroughly mix the contents of the urn7 select a chip and record the letter7 and replace the chip to the urn77 are repeated until an a is recorded The sample space for this experiment is the countably in nite set 9 1 bot7 bba7 bbbo L7 bbbba7 a Find the probability that an a is recorded in 4 or fewer repetitions 1 Find the probability that an a is recorded in 4 or more repetitions 13 Counting Methods 131 Multiplication Rule Sampling WithWithout Replacement Methods for counting the number of elements in a sample space or event are important in probability The multiplication rule is the basic counting method Theorem Multiplication Rule If an operation consists of r steps of which 0 the rst can be done in n1 ways7 o for each of these the second can be done in n2 ways7 o for each of the rst and second steps the third can be done in n3 ways7 o and so forth7 then the entire operation can be done in n1 x n2 x x n ways Two important special cases of the multiplication rule are 1 Sampling With Replacement For a set of size n and a sample of size r there are a total of n7nxnxxn ordered samples7 if duplication is allowed to Sampling Without Replacement For a set of size n and a sample of size r there are a total of nl Wnxnilxxn77l ordered samples7 if duplication is not allowed Note that the symbol n read n factorial is de ned as n nn 71n 7 2 1 when n is a positive integer and 0 1 Exercise 1 Choice of one 2 Choice of one 3 Choice of one 4 Choice of one 5 Choice of one A menu in a restaurant reads like this chicken soup7 tomato juice7 fruit cocktail beef hash7 roast ham7 fried chicken7 spaghetti with meat balls potatoes7 brocolli7 lima beans chocolate ice cream7 apple pie coffee7 tea7 milk a Find the number of complete dinners7 where a complete dinner consists of one choice from each category 1 Suppose that you can t choose both spaghetti with meatballs and potatoes How many complete dinners are there now 0 Suppose that the choice none is added to each category Find the number of dinners with at least one item Find the number of dinners with at least two items Exercise The members of the executive committee of your club must decide among them selves Who will serve as President7 Vice President7 Secretary and Treasurer with the remaining individuals serving as just committee members How many different assignments are there a if there are a total of 4 people on the executive committee 1 if there are a total of 8 people on the executive committee EZETCZSE Suppose that you are in charge of of scheduling speakers for the student association There are ve dates Ten people are interested in speaking Bob Alice Ted Jim Mary Fred Oscar Beth Charlie Chris A schedule is a matching of ve di erent people to the ve dates 0 Find the number of schedules 1 Find the number of schedules if Bob can only come on one of the rst three dates Example The Birthday Problem Suppose that there are r unrelated people in a room none of whom was born on February 29th of a leap year You would like to determine the probability that at least 2 people have the same birthday You ask and record each person7s birthday There are 365T possible outcomes where an outcome is a sequences of 1 responses Consider the event everyone has a di erent birthday77 The number of outcomes in this event is m365x364xx3657r1 Suppose that each sequence of birthdays is equally likely The probability that at least two eople have a common birthday is 1 minus the probability that everyone has a di erent birthday or 365x364xnx 3657r1 muck 365 X where AT is the event that at least two people have a common birthday when there are r people in the room Hm It is surprising how quickly the probability that at least two people have a common birthday grows un der these simplistic assumptions For example 0 PAT gt 050 when r 2 23 o PAT gt 099 when r 2 57 Exercise Using the same simplifying assumptions as above nd a general formula for the probability that at least one of the r individuals was born on November 7th and evaluate your formula when r 30 132 Permutations Combinations Binomial Coef cients Binomial Theorem 0 A permutation is an ordered subset of r distinct objects out of a set of n objects 0 A combination is an unordered subset of r distinct objects out of the n objects Counting rules By the multiplication rule there are a total of nl Wnxn71xxn771 nP39r permutations of r objects out of n objects Since each unordered subset corresponds to r ordered subsets the r chosen elements are permuted in all possible ways there are a total of PT nl nxn71xxnir1 C n7 Tl nir rl TgtltltT71gtltgtlt1 combinations of r objects out of n objects For example there are 6P3 6 x 5x4 120 ordered subsets of3 elements from 1 2 3 4 5 6 The 120 ordered subsets can be represented as follows 123 124 125 126 132 134 135 136 142 143 145 146 152 153 154 156 162 163 164 165 213 214 215 216 231 234 235 236 241 243 245 246 251 253 254 256 261 263 264 265 312 314 315 316 321 324 325 326 341 342 345 346 351 352 354 356 361 362 364 365 412 413 415 416 421 423 425 426 431 432 435 436 451 452 453 456 461 462 463 465 512 513 514 516 521 523 524 526 531 532 534 536 541 542 543 546 561 562 563 564 612 613 614 615 621 623 624 625 631 632 634 635 641 642 643 645 651 652 653 654 Each unordered subset corresponds to 3 6 ordered subsets implying that there are 20 unordered subsets 123 124 125 126 134 135 136 145 146 17576 27374 27375 27376 27475 27476 27576 37475 37476 37576 47576 15 The notation read 71 choose r is used to denote the total number of combinations Special cases are as follows ltgtltgt1 mg Further7 since choosing r elements to form a subset is equivalent to choosing the remaining 71 7 r elements to form the complementary subset7 ltngtlt n forr01n 7 7177 Binomial coef cients The quantities r 017717 are often referred to as the binomial coe cients because of the following theorem Theorem Binomial Theorem For all numbers z and y and each positive integer n 90 y g y Idea of the proof The product on the left can be written as a sequence of n factors 90y 90yX 00yX X 1y The product expands to 2 summands7 where each summand is a sequence of 71 letters one from each factor For each r exactly sequences have T copies of z and n 7 r copies of y Exercise Use the binomial theorem to demonstrate that a set with n elements has 2 subsets Exercise You toss a fair coin n times and record h for head or t for tail each time Let A be the event that there are exactly r heads in the sequence Find a general formula for PATT0171 and evaluate your formula when n 8 and r 3 133 Simple Urn Model Maximum Likelihood Estimation Suppose there are M special objects in an urn containing a total of N objects In a subset of size n chosen from the urn exactly m are special Unordered Subsets There are a total of M X N 7 M m 71 7 m unordered subsets with exactly m special objects and exactly 71 7 m other objects If each choice of subset is equally likely then for each m if X N M nim IX Pm special objects Ordered Subsets There are a total of n ltgtX MPm gtlt NiMPnim m ordered subsets with exactly m special objects The positions of the special objects are selected rst followed by the special objects to ll these positions followed by the non special objects to ll the remaining positions If each choice of subset is equally likely then for each m gtlt MPm gtlt NiMPnim Pm special objects P N n lnterestingly Pm special objects is the same in both cases For example let N 25 M 10 n 8 and m 3 Then using the rst formula W 7 120 x 3003 7 728 7 7 x 0333 285 1081575 2185 P3 special objects Using the second formula the probability is 3 x loPg x 15P5 i 56 x 720 x 360360 7 728 N 0 333 25138 43609104000 2185 N 39 39 P3 special objects Exercise A shipment of 20 laptop computers contains 4 defective and 16 good machines You select 3 machines at random and test them a How many subsets of3 contain exactly k defective machines for k O 1 2 3 eac c 01ceo su set1sequa y 1 eyW at1st epro a 11tyt att esu set blf hh fb lll kl h h bb l h h b you choose has no defective machines Estimating unknown quantities maximum likelihood estimation In statistics ins formation from a sample is used when complete information is impossible to obtain ob ai or would take too long to gather One of the most important techniques is the method of maximum likelihood developed by the British statistician RA Fisher An interesting application of Fishers method uses probabilities derived from the simple u1n model to estimate the total size of a population N based on information from two overlapping but incomplete sources 1 First some Information from a rst rnain source is used to identify asubpopulation of interest Whose size is 2 second source Information from a second independent source is used to identify a sample from the population Whose size is p 3 aperiop Individuals identi ed using both sources form the overlap Let m be the size of the overlap Let LikN read likelihood of N 77 be the probability of observing m individuals in the overlap if the true population size is N w m LikN ll The likelihood function is maximized when N M Example Regal 8 Hook Biometrics 1999 5512414 6 Spina bi da is a rare spinal column defect that can be treated but not cured Treatments for spina bi da include surgery medicas tion and physical therapy This example considers estimating the number of babies born with spina bi da in upstate New York between 1969 and 1974 using information from birth and death certi cates as the primary source of information on the disorder and rehabilitation les as the secondary source he researchers identi ed M 566 cases of spina bi da by examining the birth and death certi cates of all live births in upstate New York between 1969 and 1974 Rehabilitation les included information on 7L 188 cases with an overlap of m 128 cases Since M 7 m 438 cases were identi ed from birth and death certi cates only and 71 m 60 cases were identi ed from rehabilitation les only the sources give incomplete information about the total number of spina bi da cases The likelihood function is 566 N7566 IllMN 128 60 l N 188 The plot on the right shows LikN for values of N between 650 and 950 The function is maxi mized when 188 N 7 566 7 831 134 Partitioning Sets Multinomial Coef cients Multinomial Theorem The multiplication rule can be used to nd the number of partitions of a set of n elements into k distinguishable subsets of sizes r1 r2 77 Theorem Number of Partitions The number of ways that 71 objects can be grouped into k distinguishable subsets of sizes r1 r2 m is lt n gt 7 nl T1T2Tk T1lT2lTkl7 where the symbol on the left is read 71 choose r1 r2 rk Proof The choices are made in k steps 71 of the n elements are chosen for the rst subset r2 of the remaining 71 7 n elements are chosen for the second subset and so forth The result is the product of the numbers of ways to perform each step 71 min nini7rk4 nl X XX 71 72 m Tll Tgl rkl after simpli cation Exercise A class of 15 students is to be split into 3 recitation sections of 5 students each led by Sally Mary and Joe respectively The recitation sections are distinguished by their group leaders The total number of ways in which this can be done is l 15 i 756756 5 5 5 5x 5x 5x Recalculate this total using the multiplication rule as described in the proof above Permutations of indistinguishable objects The formula above also represents the num ber of ways to permute n objects7 where the rst 71 are indistinguishable the next r2 are indistinguishable 7 the last m are indistinguishable The computation is done as follows 71 of the 71 positions are chosen for the rst type of object7 r2 of the remaining 71 7 71 positions are chosen for the second type of object7 and so forth Exercise Find the number of permutations of the eleven letters in the word MISSISSIPPI How many permutations are there whose rst and last letters are I Multinomial coef cients The quantities Tl Wm M are often referred to as the multina mz39al coe cients because of the following theorem Theorem Multinomial Theorem For all numbers 1727 wk and each positive in teger n n 12 kn Z ilmg2Wiskv T17T277Tk Tmzwm where the sum is over all k tuples of non negative integers with n n Idea of proof The product on the left can be written as a sequence of n factors z1x2zk z1x2zkx m1m2mk X WX z1z2zk The product expands to k summands7 where each summand is a sequence of 71 letters one from each factor For each r1 r2 7 m exactly sequences have 71 copies of 1 r2 copies of 2 etc 71 7 1 7 2yw39rk 14 Conditional Probability Assume that A and B are events7 and that PB gt 0 Then the conditional probability of A given B is de ned as follows Puma P A B lt w gt 133 Notes 0 Event B is often referred to as the conditional sample space 0 The conditional probability PAB is the relative size of A within B o lf 9 is a nite set and each outcome is equally likely7 then PAB W gf Exercise Each of 1000 adults is classi ed by two criteria whether or not the person smokes S7 and whether or not the person has a respiratory disease R7 as shown on the right The name of one person is chosen at random Assuming each choice is equally likely7 H Hm PSR PRS MSW HEW Exercise Answer the questions on the right 1314 3 based on the following table of probabilities for events A and B PBA AlA U B PAA m B HAOBMUB 141 Multiplication Rule For Probability Assume that A and B are events with positive probability Then the de nition of conditional probability implies that the probability of the intersection A O B can be written as a product of probabilities in two different ways PA B PB x PAlB PA x PBlA More generally Theorem Multiplication Rule for Probability lf A1 A2 Ak are events and PA1 Az Akil gt 0 then PA1 A2 39 Ak gtlt gtlt 7142 gtlt gtlt 7142 m Ak1 Exercise Fifteen percent of the people in a certain population have respiratory disease 0 Among those with respiratory disease 45 are current smokers 35 are past smokers and 20 have never smoked cigarettes 0 Among those with no respiratory disease 20 are current smokers 30 are past smokers and 50 have never smoked cigarettes A person s name is chosen Assuming each choice is equally likely use the information above to complete the following table of probabilities Smoker Smoker Smoked Disease Disease Exercise On a rst try7 a sharpshooter can hit the bull s eye 90 of the time Given a hit7 the probability that she hits on the second try is 092 Given a miss7 the probability that she hits on the second try is 088 Fill in the following table of probabilities Event Probability The sharpshooter hits the tar get on both the rst and sec ond tries The sharpshooter hits the tar get on the rst try but not on the second try The sharpshooter hits the tar get on the second try but not on the rst try The sharpshooter misses the target on both the rst and second tries Exercise Four slips of paper are sampled without replacement from a well mixed urn con taining twenty ve slips of paper fteen slips with the letter pritten on each and ten slips of paper with the letter eritten on each Then 15 10 14 13 PXYXX PX gtlt PYlX gtlt PXlXY gtlt PXlXYX g X Q X g X E N 0109 Fill in the following table of probabilities Event Probability The sequence XXXY is chosen Exactly 1 Y and 3 X s are chosen The sequence XXYY is chosen Exactly 2 Y s and 2 X s are chosen Fill in the following table of probabilities if sampling is done with replacement Event Probability Exactly 1 Y and 3 X s are chosen Exactly 2 Y s and 2 X s are chosen 142 Law of Total Probability Law of Average Conditional Probability The law of total probability can be used to write an unconditional probability as the weighted aVerage of conditional probabilities Speci cally Theorem Law of Total Probability Let A1 A2 Ak and B be events with nonizero probability If A1 A2 Ak are pairwise disjoint with union 9 then PB PA1 x PBlA1 PA2 x PBlA2 PAk x PBlAk Proof If A1 A2 Ak are pairwise disjoint quot quotZ quot5 4 with union 9 then the sets B A1B A2B Ak B are pairwise disjoint with union B as illustrated on the right using k 4 sets to partition 9 Thus axiom 3 and the de nition of conditional probability imply that 133 im m A7 PA7PBlA7 71 71 EZETCZSE Fifteen percent of the people in a certain population haVe respiratory disease ong those with respiratory disease 45 are current smokers Among those with no respii ratory disease 20 are current smokers Find the probability that a person in this population is a current smoker EZETCZSE A city is comprised of four di erent ethnic groups 35 of the population is of ethnic group 1 25 of ethnic group 2 and 20 each of ethnic groups 3 and 4 30 50 40 and 45 of ethnic groups 1 2 3 and 4 respectively haVe blood type 0 Find the probability that a person liVing in the city has blood type 0 Law of average conditional probability The law of total probability is often called the law of average conditional probabilities PB is the weighted average of the conditional probabilities PBlA17 PBlA27 PBlAk7 where PAl7 PAg7 PAk are the weights in the weighted average 143 Bayes Rule Prior and Posterior Probabilities Diagnostic Tests Bayes rule7 proven by the Reverend T Bayes in the 1760 s7 can be used to update probabilities given that an event has occurred Speci cally7 Theorem Bayes Rule Let Al7 Ag7 Ak and B be events with non zero probability lf Al7 A2 Ak are pairwise disjoint with union 9 then P A P B A PAlB k 7 X l 7 j12k 81 PAi X HEW Proof The numerator of the expression on the right is PAj B7 and the denominator is PB by the law of total probability Thus7 the ratio simpli es to PAle Exercise A city is comprised of four different ethnic groups 35 of the population is of ethnic group 17 25 of ethnic group 27 and 20 each of ethnic groups 3 and 4 257 357 35 and 30 of ethnic groups 17 27 37 and 47 respectively7 have blood type B If a person has blood type B7 nd the probabilities that she is in ethnic groups 17 27 37 4 Prior and posterior events In applications the collection of probabilities PAj are often referred to as the prior probabilities the probabilities before observing an outcome in B and the collection of probabilities PAj are often referred to as the posterior probabilities the probabilities after event B has occurred Diagnostic screening tests In diagnostic screening a test is applied to an individual who has not yet exhibited clinical symptoms of a particular disease with the goal of determining that individuals probability of having the disease Those who test positive are considered more likely to have the disease and may be recommended to take further tests or to start treatment Let D be the event that an individual has the disease D0 be the event that the individual is disease free POS be the event the individual tests positive for disease and NEG be the event the individual tests negative for disease Then 1 a quot39 quot The 39t39 39t or tr p it rate of the test is the probability that an individual chosen at random from those with disease tests positive for disease Sensitivity PPOSlD to Speci city The speci city or true negative rate of the test is the probability that an individual chosen at random from those who are disease free tests negative for disease Speci city PNEGlDC OJ Prevalence The prevalence of the disease is the probability that an individual chosen at random from the study population has the disease Prevalence PD q Predictive Value of Positive Test The predictive value of a positive test or the positive predictive value of the test is the probability that an individual chosen at random from those testing positive for disease actually has the disease Positive Predictive Value PDlPOS U Predictive Value of Negative Test The predictive value of a negative test or the negative predictive value of the test is the probability that an individual chosen at ran dom from those testing negative for disease is actually disease free Negative Predictive Value PDclNEG Example Carpal Tunnel Syndrome Pagano 55 Gauureau 2000 page 157 Carpal tunnel syndrome is an af iction of the wrist which occurs when the median nerve a nerve running from the forearm into the hand becomes pressed or squeezed The risk of developing carpal tunnel syndrome is not con ned to individuals in a single industry but is especially common in those performing assembly line work The National Institute for Occupational Safety and Health NlOSH has established a de nition of this disorder that incorporates three criteria symptoms of nerve damage a history of occupational risk factors and the presence of physical exam ndings The sensitivity of this de nition as a test for the syndrome is 067 and the speci city is 058 That is PPOSlD 067 and PNEGD0 058 where D is the event that a worker has carpal tunnel syndrome Dc is the event that a worker does not have the disease P05 is the event that the worker satis es all three criteria established by NlOSH and NEG is the event that the worker does not satisfy all three criteria Suppose that 20 of workers in a particular industry have carpal tunnel syndrome Then the following table summarizes the calculations needed to nd the predictive value of a positive test and of a negative test Analysis of Carpal Tunnel Syndrome Test When 20 of Workers Have the Disease Predictive Value of Positive Test is 02851 Predictive Value of Negative Test is 08755 15 Independent Events EventsAandBaresaidtobe39 J J or 7 39739 quot 39Jr J if 1 PA Q B PA x PB Otherwise7 A and B are said to be dependent Theorem Independent Events Let A and B be events whose probabilities satisfy the following inequalities 0 lt PA lt 1 and 0 lt PB lt 1 Then A and B are independent ltgt PAlB PA ltgt PBlA PB Exercise Demonstrate the rst equivalence in the theorem above Exercise One chip is drawn from each of two well mixed urns The rst urn contains 5 red7 4 blue and 6 green chips The second urn contains 7 red7 3 blue and 10 green chips a The probability that both chips are red is b The probability that both chips are the same color is 151 Mutually Independent Events Events A1 A2 Ak are said to be mutually independent if c For each pair of distinct indices 2122 PAll O A PAi1 x PA2 o For each triple of distinct indices 2122233 13141391 1412 Aia 13141391X 13141392X PAi37 o and so forth Exercise Two fair six sided dice one red and one green are rolled and the number on the top face of each die is recorded The sample space can be represented as follows a 11 12 13 14 15 16 21 22 23 24 25 263132 33 343536 41 42 43 44 45 46 51 52 53 54 55 56 61 62 63 64 65 66 Let A be the event that the red die shows a 3 4 or 5 B be the event that the green die shows a 1 or 2 and C be the event that the dice total is 7 Are A B and C mutually independent Why 152 Repeated Trials and Mutual Independence As stated at the beginning of this notebook the term experiment is used in probability theory to describe a procedure whose outcome is not known in advance with certainty Experiments are assumed to be repeatable and to have a well de ned set of outcomes Repeated trials are repetitions of an experiment using the speci ed procedure with the outcomes of the trials haVing no in uence on one another The results of repeated trials of an experiment are mutually independent Exercise The following procedure is repeated six times thoroughly shuf e a standard deck of 52 cards choose a card note its suit C D H S replace the card Find the probability of obtaining a The sequence CDHDSS b Exactly two diamonds and two spades c Exactly two diamonds two spades and two hearts MT426 Notebook 2 prepared by Professor Jenny Baglivo Copyright 2009 by Jenny Ar Baglivor All Rights Reserved 2 MT426 Notebook 2 21 Introduction r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 2 1 1 De nitions Discrete Random Variables 2 21 Probability Distribution PDF CDF r r r r r r r r r r r r r r r r r r r r r r r r r Discrete Uniform Distribution r r r r r r r r r r r r r r r r r r r r r r r r r r r r r Hypergeometric Distribution r r r r r r r r r r r r r r r r r r r r r r r r r r r r r Bernoulli Experiments Bernoulli and Binomial Distributions r r r r r r r r r r r Simple Random Samples Binomial Approximation Survey Analysis r r r r r r r Geometric and Negative Binomial Distributions r r r r r r r r r r r r r r r r r r r Poisson Limit Theorem Poisson Distribution r r r r r r r r r r r r r r r r r r r r Continuous Random Variables r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r Probability Distribution PDF and CDF r r r r r r r r r r r r r r r r r r r r r r r Quantiles Percentiles r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r Continuous Uniform Distribution r r r r r r r r r r r r r r r r r r r r r r r r r r r Exponential Distribution Relationship to Poisson Process r r r r r r r r r r r r r Euler Gamma Function Gamma Distribution r r r r r r r r r r r r r r r r r r r r Distributions Related to Poisson Processes r r r r r r r r r r r r r r r r r r r r r Cauchy Distribution r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r Normal Gaussian Distribution r r r r r r r r r r r r r r r r r r r r r r r r r r r r Transforming Random Variables List of Tables Standard normal cumulative probabilities z PZ S 2 when 2 2 0 r r r r r r r r r 29 31 2 MT426 Notebook 2 This notebook is concerned with discrete and continuous random variables The notes corre spond to material covered in Chapter 2 of the Rice textbook 21 Introduction Researchers use random variables to describe the numerical results of experiments 211 De nitions 1 A random variable is a function from the sample space of an experiment to the real numbers The range of a random variable is the set of values the random variable assumes Random variables are usually denoted by capital letters X Y Z and their values by lower case letters x y 2 to If the range of a random variable is a nite or countably in nite set then the random variable is said to be discrete if the range is an interval or a union of intervals then the random variable is said to be continuous otherwise the random variable is said to be of mixed type Example 1 You toss a fair coin eight times and record the sequence of heads and tails Let X be the number of heads in the sequence and Y be the number of heads minus the number of tails Then 0 X is a discrete random variable with range 0 1 2 3 4 5 67 8 0 Y is a discrete random variable with range 78 76 74 720 2 4 6 8 and YX787X2X78 Example 2 You toss a fair coin until you get a tail and record the sequence of outcomes Let X be the trial number of the rst tail and Y be the number of heads before the rst tail Then 0 X is a discrete random variable with range 1 2 3 0 Y is a discrete random variable with range 0 1 2 and 0 Y X 7 1 Example 5 Let X be the height of an individual measured in inches with in nite precision Then X is a continuous random variable Whose range is the positive real numbers If Y is the height measured in feet with in nite precision then Y is a continuous random variable Whose range is the positive real numbers and Y EX 22 Discrete Random Variables A discrete random variable is one whose range is a nite set or a countably in nite set 221 Probability Distribution PDF CDF If X is discrete then PX z is the probability that an outcome has value m and the collection of values PX m for all real numbers m is its probability distribution Similarly PX z is the probability that an outcome has value x or less Pa lt X lt b is the probability that an outcome has value strictly between a and b and so forth Probability density function le is discrete then the probability density function PDF or frequency function FF of X is de ned as follows pz PX z for all real numbers m Probability density functions satisfy the following properties 1 Nonnegati39ue pz 2 0 for all real numbers m 2 Unit Sum Z pz 1 where R is the range of X 1673 Note that the events X z for z E R are pairwise disjoint sets whose union is the sample space of the experiment Events must have nonnegative probability property 1 and the sum of their probabilities must be the probability of the sample space property 2 Example 1 continued Let X be the number of heads obtained in eight tosses of a fair coin Then a general formula for the PDF of X is px when x 01 8 and 0 otherwise Let Y be the number of heads minus the number of tails Then the nonzero values of the PDF of Y are displayed in the following table y 8 6 4 2 0 2 4 6 8 My and PY 2 3 Cumulative distribution function The cumulative distnbutim functim ODF of the discrete random variable X is de ned as follows Fz PX g z for all rea1 numbers a Cumulative distribution functions satisfy the fo11owing properties 139 In nite Limits hm Fz U and lim Fz 1V Hroo Moo 239 Nondecmasing If 1 1 g 22 then Fz1 g Fzg 339 Right Continuous 1im Fz Fa for each real number at wan Note that Fz represents curnu1atiye probabi1ity with hrnits 0 and 1 property 1 Curnu1atiye probabi1ity increases with increasing a property 2 and has discrete jumps at wa1ues of z in the range of the random variab1e property 3 Eromple 1 cantinuad Let X be the number of heads in eight tosses of a fair coin Furs 1 Left Plot The 1eft p1ot is a pmbability histagmm of the PDF of X A probabihty histograrn is constructed as follows For each a in the range of X a rectangle with base corresponding to the interval 27050 mono and with height pc is drawn The tota1 area is 1 m Right Plat The right p1ot shows the ODF of X Fz is a step function with steps of height pc for each a in the range Since the range of X is the nite set 012 8 Fz 0 when a lt 0 and Fz 1 when a 2 s 222 Discrete Uniform Distribution Let n be a positive integer The random variable X is said to be a discrete uniform random variable or to have a discrete uniform distribution with parameter n when its PDF is as follows 1 pz g when x 12n and 0 otherwise For example suppose that you roll a fair sixesided die and let X be the number of dots on the top face Then X is a discrete uniform random variable with n 6 The PDF and CDF of X are displayed below Example Hand et al Chapman 6 Hall 1994 page 98 Is a fair die really fair That7s the question that RWolf tried to answer when he rolled a fair die 20000 times and recorded the number of dots on the top face each time The following table summarizes the results of his experiment where the expected frequency is Expected Frequency 20000 pz 20000 N 333333 for each x and the relative error is the following ratio Observed Frequency 7 Expected Frequency Rel t39 E a we or Expected Frequency Note that the relative error has been converted to a percentage x1 x2 x3 x4 x5 x6 Observed Frequency 3407 3631 3176 2916 3448 3422 Expected Frequency 333333 333333 333333 333333 333333 333333 Relative Error 221 893 7472 71252 344 266 The rather large relative errors call the fairness of the die into question 223 Hypergeometric Distribution Let n M and N be integers with 0 lt M lt N and 0 lt n lt N The random variable X is said to be a hypergeometric random variable or to have a hypergeometric distribution with parameters n M and N when its PDF is as follows when x is an integer between max0 n M 7 N and minn M and 0 otherwise Note that hypergeometric distributions are used to model urn experiments where N is the number of objects in the urn M is the number of special objects n is the size of the subset chosen from the urn and X is the number of special objects in the chosen subset If the choice of each subset is equally likely then X has a hypergeometric distribution Example A group of twenty ve rst graders is to be randomly assigned to two classes a class of 10 students to be taught by Mrs Smith and a class of 15 students to be taught by Mr Jones Assume that each choice of rosters is equally likely Five of the rst graders are close friends Let X be the number of close friends assigned to Mrs Smith s class Then X has a hypergeometric distribution where n M andN The PDF and CDF of X are displayed below o The probability that all 5 friends are assigned to the same class is o The probability that exactly 4 of the 5 friends are assigned to the same class is 224 Bernoulli Experiments Bernoulli and Binomial Distributions A Bernoulli experiment is a random experiment with two possible outcomes The outcome of chief interest is often called success and the other outcome failure Let p PSuccess and 17 p PFailurel Let X be the number of successes in n independent trials of a Bernoulli experiment with success probability p Then X is said to be a binomial random variable or to have a binomial distribution with parameters n and p The PDF of X is as follows pz paC 1717 706 when x 012Hln and 0 otherwise A Bernoulli distribution is the special case when n 1 Example You roll a fair sixsided die ve times and observe the number of dots on the top face each time Let X be the number of times you observe either a l or a 4 in the ve trialsl Then X is a binomial random variable with parameters n andp The PDF and CDF of X are displayed below o The probability of observing at least 3 successes in the 5 trials is o The probability of observing at least one success and at least one failure in the 5 trials is 225 Simple Random Samples Binomial Approximation Survey Analysis Suppose that an urn contains N objects A simple random sample of size n is a sequence of 71 objects chosen without replacement from the urn where the choice of each sequence is equally likely Let M be the number of special objects in the urn and X be the number of special objects in a simple random sample of size n Then X has a hypergeometric distribution with parameters n M N Further if N is very large then binomial probabilities can be used to approximate hypergeometric probabilities Theorem Binomial Approximation If N is large then the binomial distribution with parameters 71 and p MN can be used to approximate the hypergeometric distribution with parameters n M N Speci cally At 1224 Pm special objects T x n 19117 p for each m n 9 Note that if X is the number of special objects in a sequence of 71 objects chosen with replace ment from the urn and if the choice of each sequence is equally likely then X has a binomial distribution with parameters 71 and p MN The theorem says that if N is large then the model where sampling is done with replacement can be used to approximate the model where sampling is done without replacement Survey analysis Simple random samples are used in surveys If the survey population is small then hypergeometric distributions are used to analyze the results If the survey population is large then binomial distributions are used to analyze the results even though each person s opinion is solicited at most once For example suppose that a surveyor is interested in determining the level of support for a proposal to change the local tax structure and decides to choose a simple random sample of size 10 from the registered voter list o If there are a total of 120 registered voters onethird of whom support the proposal then the probability that exactly 3 of the 10 chosen voters support the proposal is 40 80 MM tame 10 o If there are thousands of registered voters the probability is weGrew Note that in the approximation you do not need to know the exact number of registered voters 226 Geometric and Negative Binomial Distributions Let X be the trial number of the rth success in a sequence of independent Bernoulli trials with success probability p Then X is said to be a negative binomial random variable or to have a negative binomial distribution with parameters r and p The PDF of X is as follows pz 1 7p rpr when x r r 1r 2 and 0 otherwise Note that for each m pz is the probability of the event Exactly z 7 r failures and r successes in z trials with the last trial a success77 The geometric distribution corresponds to the special case where r 1 If X has a geometric distribution then its PDF is as follows pz 1 7pm71p when x 1 2 3 and 0 otherwise Exercise Let X be the trial number of the rst success in a sequence of independent Bernoulli trials with success probability p Find simpli ed formulas for a the cumulative probability PX m where z is a nonnegative integer and b the upper tail probability PX gt m where z is a nonnegative integer Exercise A Bernoulli experiment consists of rolling a fair sixsided die and recording an S for success if 1 or 4 dots appear on the top face7 and an F for failure otherwise a Let X be the trial number of the rst success in a sequence of independent trials of the experiment Then X has a geometric distribution With parameter p The PDF and CDF of X are displayed below o The probability of observing 5 or more failures before the rst success is 17 Let X be the trial number of the third success in a sequence of independent trials of the experiment Then X has a negatiVe binomial distribution With parameters 7quot andp The PDF and CDF of X are displayed below o The probability of obserVing 4 or feWer failures before the third success is 227 Poisson Limit Theorem Poisson Distribution The following limit theorem proven by SPoisson in the 1830 s can be used to estimate bino mial probabilities when the number of trials is large and the probability of success is small Theorem Poisson Limit Theorem Let A lambda be a xed positive real number let n be an integer greater than A and let p A Then 7L A1 lim ltngtpw17pnw 547 whenz012 L700 x ml V L Note that the proof uses the fact that lim lt1 gt e7 for any real number r TLHOO 77 For example let X be the number of successes in 10000 independent trials of a Bernoulli experiment whose success probability is p and consider nding the probability that the binomial random variable X equals 3 0 Using the binomial PDF the probability is 10000 1 3 2499 9997 P X3 7 7 0195386 lt 3 gtlt2500gt 2500 0 Using Poisson s approximation 10000 1 3 2499 9997 43 P X3 7 7 g 47 0195367 l lt 3 gtlt2500gt 2500 5 3 since A np 4 Poisson distribution Let A be a positive real number The random variable X is said to be a Poisson random variable or to have a Poisson distribution with parameter A if its PDF is as follows Ail pz PX z e A 7 when x 012 and 0 otherwise 1 00 Exercise Use the Maclaurin series for e7 to demonstrate that 1 m0 Poisson process Events occurring in time or in space are said to be generated by an approximate Poisson process With rate A When the following three conditions are satis ed 1 Independence The number of events occurring in disjoint subintervals of time or subregions of space are independent of one another to Proportionality The probability of one event occurring in a suf ciently small subinterval of time or subregion of space is proportional to the size of the subinterval If h is the size of the subinterval or subregion then the probability is Ah on Unlimited Number of Events Any number of events are possible Within a single interval of time or region of space But the probability of tWo or more events occurring in a suf ciently small subinterval or subregion is virtually 0 In this de nition A represents the average number of events per unit time or space If events follow an approximate Poisson process and X is the number of events observed in one unit of time or space then X has a Poisson distribution With parameter A Poisson distributions have been used for example to model the number of angina attacks a patient suffers in a certain period of time the number of industrial accidents occurring in a certain period of time and the number of leukemia cases occurring Within a certain distance of a toxic Waste site Example Suppose that cars passing a certain intersection in the middle of a Work day follow a Poisson process With an average of 7 cars per hour Let X be the number of cars passing in a thirtyeminute period Then X has a Poisson distribution With A The PDF and GDP of X are displayed below PX o The probability of observing 4 or fewer cars in a thirtyeminute period in the middle of a Workday is 23 Continuous Random Variables A continuous random variable is one whose range is an interval or a union of intervals 231 Probability Distribution PDF and CDF The probability distribution of a continuous random variable is given in terms of its cumulative distribution function or its probability density function Cumulative distribution function If X is continuous then the cumulatiue distribution function CDF of X is de ned as follows PX z for all real numbers z Cumulative distribution functions satisfy the following properties 1 In nite Limits lim 0 and lim 1 maioo 1400 2 Nondccmasing lf 1 m2 then Fz1 3 Continuous is a continuous function for all real numbers m Note that represents cumulative probability with limits 0 and 1 property 1 Cumulative probability increases with increasing z property 2 For continuous random variables the CDF is a continuous function property 3 Probability density function If X is continuous then the probability density function PDF or density function of X is de ned as follows 1 fz d7 whenever the derivative exists 1 Probability density functions satisfy the following properties 1 Nonncgati39uity fz 2 0 whenever f exists 2 Unit Integral fz dz 1 where R is the range of X R Note that fz represents the rate of change of probability The rate of change of probability must be nonnegative property 1 Since fz is the derivative of the CDF the area under the graph of f must be 1 property 2 Computing probabilities using the PDF If R is the range of X and I is an interval then the probability of the event the value of X is in the interval I77 is obtained by nding the area under the density curve for z E I R PXEI made I R 14 Note7 in particular7 that if a E 73 then PX a 0 since the area under the density curve and over an interval of length zero is zero Exercise Let X be the continuous random Variable With PDF as follows 3 x12 When 71 x g 1 and 0 otherwise The PDF and CDF of X are displayed below 1 Completely specify the cumulative distribution function of X b Find PX 2 0 Exercise Let X be the continuous random Variable With PDF as follows 1 00 m When 95 2 0 and 0 otherwise at The PDF and CDF of X are displayed below 1 Completely specify the cumulative distribution function of X b Find PX 2 8 0 Suppose instead that the PDF of X is x m when x 2 0 and 0 otherwise7 where c is some positive constant Find the value of 0 232 Quantiles Percentiles Let X be a continuous random variable7 and p be a proportion satisfying 0 lt p lt 1 The pth quantile or 100pth percentile of the X distribution is the point7 mp satisfying the equation PX mp p To nd mp solve the equation p for m Median quartiles interquartile range Important special cases are 1 Median The median of X is the 50th percentile 2 Quartiles The quartiles of X are the 25th7 50th7 and 75th percentiles ln addition7 the interquartile range IQR is the following difference QR 75th Percentile 7 25th Percentile Note that the median is a measure of the center of a continuous distribution7 and the in terquartile range is a measure of the spread of the distribution EZETCZSE In each case nd a general fonnula for the p h quantile the median and the in terquaitile range of the X distribution Nli a x g0 1 when a e 711 else 039 27 m wig 3 when a e 000 else 039 233 Continuous Uniform Distribution Let a and I be real numbers with o lt l The random Variable X is said to be a uniform random uariable or to have a uniform distribution on the interval 11 when its PDF is as follows fz b E a when a g I g I and 0 otherwise General forms of the PDF and CDF of X are shown below quot1 VF 7 14 i r b o b quot EZETCZSE Let X be a uniform random Variable on the interVal 11 0 Demonstrate that the probability that X lies in a subinterVal of 11 depends only on the length of the subinterVal 12 Use your answer to pan 0 to nd the median and interquartile range of X EZETCZSE Recall that positive angles are measured counterclockwise from the positive ziaxis and negative angles are measured clockwise from the positive ziaxis Let 9 be an angle in the interval 07r Given 0 6 07r construct a ray through the ori7 gin at angle 0 and let 111 be the point Where the ray intersects the halficircle of radius 1 as illustrated in the plot to the right Assume 9 is a uniform random variable on the interval 07r Find the probability that a the zicoordinate of the point of intersection is less than 12 b the yicoordinate of the point of intersection is less than 12 234 Exponential Distribution Relationship to Poisson Process Let A be a positive real num T e random Variable X is said to be an exponential random variable or to have an exponential distTibutiO with parameter A when its PDF is as follows fz Ae M when I Z 0 and 0 otherwise General forms of the PDF and CDF of X are shown below Wm quot70 himA himA Note that l have marked the location of the median in each plot EZETCZSE Let X be an exponential random Variable with parameter A 0 Completely specify the cumulative distribution function of X 12 Find a general formula for the p h quantile of the X distribution c Use your answer to part b to show that the median of the X distribution is ln2 and that the interquartile range is ln3 Relationship to Poisson process If events occurring over time follow an approximate Poisson process with rate where is the average number of events per unit time then the time between successive events has an exponential distribution with parameter To see this If you observe the process for 25 units of time and let Y equal the number of observed events then Y has a Poisson distribution with parameter t The PDF of Y is as follows t y py ei vg when y 012 and 0 otherwise y An event occurs the clock is reset to time 0 and X is the time until the next event occurs Then X is a continuous random variable whose range is z gt 0 Further PX gt t PO events in the interval 0t PY 0 e and PX t 17 5 The PDF of X can be obtained from the CDF using derivatives Since 1 d t7PXltt 7lt17 4 A f gt d lt gt d e s when t gt 0 and 0 otherwise is the same as the PDF of an exponential random variable with parameter X has an exponential distribution with parameter Example Hand et al Chapman 55 Hall 1994 Researchers in Great Britain studied the occurrences of major earthquakes worldwide over the period of 1900 to 1980 They determined that the average time between successive events was approximately 425 days and that events followed an approximate Poisson process with rate or 1 event every 425 days on average Let X equal the time between successive major earthquakes and assume that X has an exponential distribution with A Then 0 The probability that the time between successive earthquakes is between 200 days and 625 days is o The median time between successive major earthquakes is 235 Euler Gamma Function Gamma Distribution The Euler gamma functton is de ned as follows pm 00 I r zl 1e dz o If r is a positive integer then I r r 7 1 Thus the gamma function is said to tnterpolate the factorials as illustrated on the right Gamma distribution Let a and A be positiVe real numbers The continuous random Variable X is said to be a gamma random wrtable or to haVe a gamma drstrtbutton with parameters a and A when its PDF is as follows a 75 i W I a 1e M when I gt 0 and 0 otherwise Note that if a r is a positiVe integer then the PDF of X is r fz W zl 1e M when I gt 0 and 0 otherwise r 7 Further if r 1 then X has an exponential distribution with parameter A Relationship to Poisson process If events occurring over time follow an approximate Poisson process with rate where is the average number of events per unit time and if r is a positive integer then the time until the rth event occurs has a gamma distribution with parameters 04 r and To see this If you observe the process for 25 units of time and let Y equal the number of observed events then Y has a Poisson distribution with parameter t The PDF of Y is as follows 7 67M AW 7 7y py when y 012 and 0 otherwise Let X be the time you observe the rth event starting from time 0 Then X is a continuous random variable whose range is z gt 0 Further PX gt t 771 Pfewer than T events in the interval 0 25 PY lt r Z 5 y 110 and PX t 17 PX gt t d The PDF ft PX t is computed using the product rule 1 771 771 d M T My 4 My 4 Ayyty l 7 1 7 e Z 7 e Z 7 7 e Z 7 dt 210 yl 210 yl gt 211 yl gt 771 771 yty Ayiltyil A57 2 7 7 Z 7 l 7 l KM y gt F y 1 gtl Ariltril 7 7At 5 l r 71 l Since the PDF of X is the same as the PDF of a gamma random variable with parameters 04 r and X has a gamma distribution with parameters 04 r and Note that step implies that if r is a positive integer then the GDP of a gamma random variable with parameters 04 r and is y PX z 17 57 when x gt 0 and 0 otherwise y 110 771 and the upper tail probability is PX gt z Z 5 when x gt O y 110 24 Example Suppose that cars passing a certain intersection in the middle of a work day follow a Poisson process7 with an average of 5 cars per hour Let X be the time in hours until the third car passesi Then X has a gamma distribution with parameters ar7andk o The probability that three cars will pass the intersection in one hour or less is o The probability that it takes more than 45 minutes for three cars to pass the intersection is 236 Distributions Related to Poisson Processes In summary there are three distributions related to Poisson processes over time 1 03 Poisson distribution If X is the number of events occurring in a xed period of time then X is a Poisson random variable with parameter A where A equals the average number of events for that xed period of time The probability that exactly z events occur in that interval is when x O 1 2 and 0 otherwise Am 7 7 7 PX 7 m 7 e g Ewponential distribution If X is the time between successive events then X is an exponential random variable with parameter A where A is the average number of events per unit time The GDP of X is 1 7 5 when x gt 0 and 0 otherwise Gamma distribution If X is the time to the rth event then X is a gamma random variable with parameters 04 r and A where A is the average number of events per unit time The GDP of X is r71 7A1 17 Z 5 0 when x gt 0 and 0 otherwise 237 Cauchy Distribution Let a be a real number and I be a positiVe real number The continuous random Variable X is said to be a Cauchy random variable or to have a Cauchy dZSthutw with center a and spread 12 when its PDF and CDF are as follows 10 hPDF39ff all al b am 1 z 7 W52zia2 or re num ers z 170 2 Cauchy CDF Fz 71rtan 1 lt gt for all real numbers I General forms of the PDF and CDF of X are shown below Vfl Fro m Note that l have marked the locations of the quaJtiles of the distribution EZETCZSE Let X be a Cauchy random Variable with center a and spread 17 0 Find a general formula for the p h quantile of the X distribution 12 Use your answer to pan 0 to demonstrate that the median of the X distri7 bution is a and the interquaJtile range is 217 Exercise Recall that positive angles are mea7 sured counterclockwise from the positive xiaxis and negative angles are measured clockwise from the positive xiaxis Let G be an angle in the interval 7 Given 9 6 7 g construct a ray through the origin at angle 9 and let Ly be the point where the ray intersects the line ac 1 as illustrated in the plot to the right Assume G is a uniform random variable on the interval 7 g and let Y be the yicoordinate of the point of intersection as described above a Demonstrate that PY S y g ltan 1y for all 1 7r 7 Use your answer to part a to conclude that Y has a Cauchy distribution 238 Normal Gaussian Distribution Let n be a real number and o be a positive real number The continuous random variable X is said to be a normal ranclom wrmble or to have a normal clrstrrbutron with mean n and stande deviation 0 when its PDF and CDF are as follows 1 Normal PDF fz e t 2lt2 2 for all real numbers I 1 V27 0 2 Normal CDF Fz I I 7 M for all real numbers I where I is an important function of applied mathematics that we will discuss below Here are general forms of the PDF and CDF of X vFx Fm Note that l have marked the locations of the quartiles of the distribution Gaussian distribution The normal distribution is also called the Gaussran clrstrrbutron in honor of the mathematician Carl Friedrich Gauss Standard normal distribution phifunctions The standard normal clrstrrbutron is the n distribution with n 0 an o 1 It is common practice to use the letter Z to represent the standard normal random variable Special letters are also used for the PDF and CDF of the stande normal random variable Z ame y 1 42 7 7 z 2 e 2andlt1gt27PZ 27700 for all real numbers 2 1 7222 7 5 dt 1 er Since 752 has no exact antiderivative the values of 2 must be approximated Table 1 page 31 gives values of the stande normal CDF when 2 Z 0 where 2 Row Value Column Value to two decimal places of accuracy Cumulative probabilities are given with four decimal place accuracy When 2 ZS negative We use the fact that the normal density curve is symmetric to nd cumui latiVe probabilities Speci cally if 2 lt 0 then M 1 7 M72 as illustrated on the right EZETCZSE Use Table 1 page 31 to nd PZ g 053 PZ g 7132 and P71 lt Z lt 15 Quantiles of the standard normal distribution Let 2 be the p h quantile of the stan7 dard normal distribution That is 2 is the point satisfying the equation zp p EZETCZSE Use Table 1 page 31 to nd approximately the 20m 40m 60Lh and 80Lh per centiles of the stande normal distribution Table 1 Standard normal cumulative probabilities z PZ g 2 when 2 Z 0 oooooooooo scoodcbmpwxw xo 0 02 0 5080 0 5478 0 5871 0 6255 0 6628 0 6985 0 7324 0 7642 0 7939 0 8212 0 8461 0 8686 0 8888 0 9066 0 9222 0 9357 0 9474 0 9573 0 9656 0 9726 0 06 0 5239 0 5636 0 6026 0 6406 0 6772 0 7123 0 7454 0 7764 0 8051 0 8315 0 8554 0 8770 0 8962 0 9131 0 9279 0 9406 0 9515 0 9608 0 9686 0 9750 0 07 0 5279 0 5675 0 6064 0 6443 0 6808 0 7157 0 7486 0 7794 0 8078 0 8340 0 8577 0 8790 0 8980 0 9147 0 9292 0 9418 0 9525 0 9616 0 9693 0 9756 NHHHHHHHH oomqmmpmm mmmmmmmmm scoodcbmpmnw x 0103030303030303 dompwmwo 38 0 9999 0 9999 0 9783 0 9830 0 9868 0 9898 0 9922 0 9941 0 9956 0 9967 0 9976 0 9982 0 9987 0 9991 0 9994 0 9995 0 9997 0 9998 0 9999 0 9999 0 9999 0 9999 0 9999 0 9999 0 9803 0 9846 0 9881 0 9909 0 9931 0 9948 0 9961 0 9971 0 9979 0 9985 0 9989 0 9992 0 9994 0 9996 0 9997 0 9998 0 9999 0 9999 0 9999 0 9808 0 9850 0 9884 0 9911 0 9932 0 9949 0 9962 0 9972 0 9979 0 9985 0 9989 0 9992 0 9995 0 9996 0 9997 0 9998 0 9999 0 9999 0 9999 0 9999 0 9999 31 Quantiles of the normal distribution Let my be the p h quantile of the normal distribu7 tion with mean u and standard deviation T Then I u 2P0 Where 2 is the p h quantile of the standard normal distribution EZETCZSE Let X be a normal random variable with mean u and stande deviation T Use Ta ble 1 page 31 to demonstrate that the median of the X distribution is u and the interquartile range is approximately 1360 EZETCZSE Agrestz 5 Franelm 2007 page 306 Distributions of heights for adult men and for adult Women are often Welliapproximated by normal distributions In North America for example 1 Women s Heights the distribution of heights for adult Women is Well approximated by a normal distribution with mean 65 inches and stande devii ation 35 inches quoti 1 4 2 Men s Heights the distribution of heights for adult men is Well approximated by a normal distribution with mean 70 inches and stande deviation 4 inches 5 w quotl 8 iAiiiniiri In each case I drew a ziaxis Where 2 z 7 MU under the ziaxis and highlighted the area under the curve between 5 feet 60 inches and 6 feet 72 inches 32 Use the normal models to a Find the probability that 1 an adult women in North America is between 5 and 6 feet tall 2 an adult man in North America is between 5 and 6 feet tall b Find the 20th 40th 60th and 80th percentiles of each distribution 239 Transforming Random Variables Let X be a continuous random variable and Y gX7 where g is a continuous function Our goal is to determine the Y distribution The rst steps will be to compute 130 S y P9X lt y as an expression in y and to nd PY y7 for each y in the range of Y Exercise Let X be the continuous random variable with PDF x 3 when 0 lt z lt 27 and 0 otherwise7 i 1 and let Y be the reciprocal of X7 Y a Find the range of Y 1 Use the X distribution to write PY y as an expression in y and nd the derivative of the expression with respect to y digPO S 0 Completely specify the PDF of Y EZETCZSE Consider a Wire of length 10 inches with a coordinate system as shown on the rig t Given 1 6 010 imagine bending the Wire at position I by 90 to form a right triangle and let y be the area of that triangle u A in Let X be an arbitra1y bending point and Y be the area of the resulting triangle Assume that X has a uniform distribution on 010 Completely specify the PDF of Y Exercise Let Z be the standard normal random variable7 and let Y Z2 Completely specify the PDF of Y MT426 Notebook 4 prepared by Professor Jenny Baglivo Copyright 2009 by Jenny Ar Baglivor All Rights Reserved 4 MT426 Notebook 4 3 41 Expected Value of a Random Variable r r r r r r r r r r r r r r r r r r r r r r r r r r r r 3 411 De nitions r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 3 412 Expectations for the Standard Models r r r r r r r r r r r r r r r r r r r r r r r r 6 42 Expected Value of a Function of a Random Variable r r r r r r r r r r r r r r r r r r r r 6 43 Properties of Expectation r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 10 4 4 Variance and Standard Deviation r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 10 4 41 De nitions and Properties r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 10 4 42 Chebyshev Inequality r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 13 4 43 Variances for the Standard Models r r r r r r r r r r r r r r r r r r r r r r r r r r 14 45 Expected Value of a Function of a Random Pair r r r r r r r r r r r r r r r r r r r r r r 14 4 51 De nitions and Properties r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 14 4 52 Covariance Correlation Association r r r r r r r r r r r r r r r r r r r r r r r r r 18 4 53 Correlations for the Standard Models r r r r r r r r r r r r r r r r r r r r r r r r 23 4 54 Conditional Expectation Regression r r r r r r r r r r r r r r r r r r r r r r r r r 24 4 55 Historical Note Regression To The Mean r r r r r r r r r r r r r r r r r r r r r r 29 46 Expected Value of a Linear Function of a Random kTuple r r r r r r r r r r r r r r r r 31 461 Mean and Variance r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 31 4 62 Covariance r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 32 463 Random Sample Sample Sum Sample Mean r r r r r r r r r r r r r r r r r r r r 35 464 Independent Normal Random Variables r r r r r r r r r r r r r r r r r r r r r r r 36 4 7 Moment Generating Function r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 37 4 MT426 Notebook 4 This notebook is concerned with mathematical expectation The notes correspond to material in Chapter 4 of the Rice textbook 41 Expected Value of a Random Variable The concept of expected value generalizes the concept of weighted average 411 De nitions 1 Discrete Case If X is a discrete random variable with range R and PDF px7 then the expected value of X is de ned as follows EX Z I PI7 673 as long as 296673 px converges If the sum does not converge absolutely7 then the expected value is said to be indeterminate 2 Continuous Case If X is a continuous random variable with range R and PDF fx7 then the expected value of X is de ned as follows 1500 Rz fltzgt dz as long as R dx converges If the integral does not converge absolutely7 then the expected value is said to be indeterminate EX is also called the expectation of X or the mean of Xi Exercise 1 Let X be the discrete random variable with PDF 7 7 x 21 7 px for x 1723745767 and 0 otherwise Compute Exercise 2 Let X be the discrete random variable With PDF 25 171 e z395 7 for z 017237Hi7 1 and 0 otherwise Compute f X 25 Exercise 5 Let X be the continuous random variable With PDF 2 for z 6 057 and 0 otherwise Compute Exercise 4 a Let X be the continuous random variable With PDF 8 W for z 2 07 and 0 otherwise Compute 12 Let X be the continuous random variable With PDF 2 W for z 2 07 and 0 otherwise Demonstrate that EX is indeterminate 412 Expectations for the Standard Models The following table gives the expectations for the standard discrete and continuous models Probability Model Expected Value Discrete Uniform Distribution on 17 27 7 n EX TH Hypergeometric Distribution based on choosing a subset of size n EX n from an urn containing M special objects and a total of N objects Bernoulli Distribution With success probability p EX p Binomial Distribution based on n trials With success probability p EX np Geometric Distribution Where X is the trial number of the first EX success in a sequence of independent trials With success probability p Negative Binomial Distribution Where X is the trial number of the r h EX success in a sequence of independent trials With success probability p Poisson Distribution With parameter A EX A Continuous Uniform Distribution on the interval a7b EX a7 Exponential Distribution With parameter A EX Gamma Distribution With shape parameter a and scale parameter A EX Cauchy Distribution With center a and spread 12 EX is indeterminate Normal Distribution With mean u and standard deviation 0 EX u 42 Expected Value of a Function of a Random Variable Let gX be a real valued function of the random variable X 1 to Discrete Case If X is a discrete random variable with range R and PDF Mm7 then the expected ualue of gX is de ned as follows E9X Z We We 1672 as long as ERR pz converges If the sum does not converge absolutely7 then the expected value is said to be indeterminate Continuous Case If X is a continuous random variable with range R and PDF u then the expected ualue of gX is de ned as follows Rgltzgt M dm as long as fR x dz converges If the integral does not converge absolutely7 then the expected value is said to be indeterminate E9X Exercise Assume that the probability of nding oil in a given drill hole is 030 and that the results nding oil or not are independent from drill hole to drill hole An oil company drills one hole at a time If they nd oil then they stop otherwise they continue However the company only has enough money to drill ve holes Let X be the number of holes drilled a Find the expected number of holes drilled b The company has borrowed 250000 to buy equipment at the rate of 12 per drilling period and have decided to pay back the loan once all drilling has been completed Thus gX 250112X is the amount in thousands of dollars due once all drilling has been completed Find the expected amount they will need to return EgX Exercise Gambler s Rum Assume that the probability of winning a game is 050 and that the results Win or lose are independent from game to game A gambler decides to play the game until she Wins Let X be the number of games played a Find the expected number of games played b The gambler places a 1 bet on the rst game and for each succeeding game doubles the bet placed of the last game Thus gX 2X 1 is the amount in dollars bet on the last game played Demonstrate that the expected amount bet on the last game EgX is indeterminate Exercise Recall that positive angles are mea7 sured counterclockwise from the positive xiaxis and negative angles are measured clockwise from the positive ziaxis Let 3 be an angle in the interval 0 Given 9 E 0 g construct a ray through the origin at angle 9 and let Ly be the point Where the ray intersects the line at 1 as illustrated in the plot to the right Assume 3 is a uniform random variable on the interval 0 g and let 99 be the yicoordinate of the point of intersection as described above 1 Find the expected yicoordinate b What value Would you get if 3 Were uniformly distributed on 0 quot7 2 43 Properties of Expectation The following properties can be proven using properties of sums and integrals H Constant Function If a is a constant then Ea a to Linear Function lf EX can be determined and a and b are constants then Ea bX a bEX 03 Linear Function of Functions lf can be determined for i 12 k and a and b are constants for i 1 2 k then k k E a Elmm a 2bEgX i391 i1 For example let X be an exponential random variable with parameter 1 Integration by parts can be used to demonstrate that EX 1 EX2 2 and EX3 6 Using these facts we know that E53X74X2X3 44 Variance and Standard Deviation The mean of X is a measure of the center of the distribution The variance and standard deviation are measures of the spread of the distribution 441 De nitions and Properties Let X be a random variable with mean u Then 1 Variance The variance of X is de ned as follows varltXgt EltltX 7 m The notation 02 VarX is used to denote the variance 2 Standard Deviation The standard deviation of X is de ned as follows SDX W The notation a SDX is used to denote the standard deviation Exercise Use the properties of expectation to demonstrate the following properties 1 VarX EX2 7 EX2 2 IfY abX for constants a and b7 then VarY bZVarX and SDY b SDX Exercise 1 page 5 continued Find the mean and variance of the discrete random variable X given in Exercise 17 page 3 Exercise 5 page 4 continued Find the mean and variance of the continuous random variable X given in Exercise 37 page 4 442 C hebyshev Inequality Chebyshev7s inequality gives us a lower bound for the probability that X is Within k standard deviations of its mean Where k is a positive constant Theorem Chebyshev Inequality Let X be a random variable With mean H EX and standard deviation 0 SDX and let k be a positive constant T en 1 P XiJ ltk0PHkUltXltJk0gtli Note Equivalently We can say that P X 7 M 2 k0 g For example consider the discrete random variable X Whose PDF is given on the left below H a rm A l For this random variable H EX 659 and 0 SDX 333 To illustrate the Chebyshev inequality let k 2 Since 720 and H20 77 the probability that X is Within 2 standard deviations of its mean is PH720ltXltJ20 This probability is greater than the guaranteed lower bound of 443 Variances for the Standard Models The following table gives the variances for the standard discrete and continuous models Probability Model Variance Discrete Uniform Distribution on 1 2 n vmltXgt i Hypergeometric Distribution with parameters n M N VarX n 1 gt Bernoulli Distribution with success probability p VWX 1017 0 Binomial Distribution based on n trials with success probability p VWX 711017 0 Geometric Distribution with parameter p VarX 1777 Negative Binomial Distribution with parameters r p VarX g Poisson Distribution with parameter A Var X A Continuous Uniform Distribution on the interval ab VarX Exponential Distribution with parameter A VarX Y1 Gamma Distribution with shape parameter a and scale parameter A VarX Cauchy Distribution with center a and spread 12 Var X is indeterminate Normal Distribution with mean a and standard deviation 0 VarX 02 45 Expected Value of a Function of a Random Pair 451 De nitions and Properties Let gX Y be a real valued function of the random pair X Y 1 Discrete Case If X and Y are discrete random variables with joint range R Q R2 and joint PDF pm y then the expected ualue of gXY is de ned as follows E9X7Y Z 9m7ypxy my 672 as long as 21 wag lgzyl pzy converges If the sum does not converge absolutely then the expected value is said to be indeterminate 2 Continuous Case If X and Y are continuous random variables with joint range R Q R2 and joint PDF u y then the expected ualue ofgX Y is de ned as follows E9X7YRgmy aw M as long as lgm u y dA converges If the integral does not converge absolutely then the expected value is said to be indeterminate 14 Properties The following properties can be proven using properties of sums and integrals 1 Linear Function of Functions lf EgXY can be determined for i 1 2 and a and bi are constants for i 1 2 k then k k E a 21ng Ygt a ZbiE9iX7 Y i1 i1 to Pmduct Function Under Independence If X and Y are independent random vari ables and gX and hY are real valued functions then Exercise Verify the 2nd property when X is a continuous random variable with range RX and PDF fXz and Y is a continuous random variable with range Ry and PDF fyy Exercise Let X and Y be the discrete random variables Whose joint distribu tion is given in the table on the right Find EX 7 Yi EZETCZSE A stick of length 5 has a coordinate system as shown below The stick is broken at random in two places Let X and Y be the locations of the two breaks and assume that X and Y are independent uniform random Variables on the interval 0 5 Find the expected length of the middle segment 452 Covariance Correlation Association Let X and Y be random variables with nite means and nite variances Then 1 Cooarrarzce The cooarrarzce of X and Y denoted by CarX Y is de ned as follows coax Y E X 7 MW 7 M where o and My EY provided the expectation exists 2 Correlatron The correlatron of X and Y denoted by CarrX Y is de ned as follows CazX Y CmX Y C X Y gt W SDXSDY VarXVorY provided the covariance exists Note that p CarrX Y is a common shorthand for the correlation and that p is often called the correlatron coe icrmt Covariance is a generalization of variance To understand covariance imagine the plane with origin many Points in the 15 and 3 01 quadrants contribute positive values and points in the 2nd and 4Lh quadrants contribute negative values to the come putation of covariance lf points in the 15 and 3 01 quadrants outweigh those in the second and fourth quadrants then the covariance will be positive the opposite will be true if the points in the 2nd and 4Lh quadrants outweigh those in the rst and third quadrants Covariance and correlation are measures of the association between two random variables Speci cally 1 Posrtwe Assocratron The random variables X and Y are said to be posrtwely assocrated if ases Y tends to increase If X and Y are positively associated then Cm X Y and CarrX Y will be positive 2 Negatwe Assocratron The random variables X and Y are said to be negatwely assocrated i f as X increases Y tends to decrease If X and Y are negatively associated then CarX Y and CarrX Y will be negative For example the height and weight of individuals in a given population are positively associ7 ated Educational level and indices of poor health are often negatively associated First list of properties Properties of covariance and correlation include the following H COUXX VarX to C390vX7 Y C390vY7 X 03 CovX Y EXY 7 EXEY F lCorMX Y 1 U 00me7 Y 1 if and only if Y a bX7 except possibly on a set of probability zero a If X and Y are independent7 then CovX7 Y 0 and CorrXY 0 Exercise Use properties of expectation to demonstrate that CovX7 Y EXY7EXEY Exercise Let X and Y be the discrete random variables Whose joint distribution is given in the table on the right Find COUXY and COTTXY Exercise Let X and Y be the continuous random variables Whose joint PDF is 2 i m y 7 z when 0 lt z lt y lt 3 and 0 otherw1se Find COUXY and COTTXY Correlated uncorrelated lf C390MX7 Y 07 then X and Y are said to be uncorrelated otherwise7 they are said to be correlated Independent random variables are uncorrelated7 but uncorrelated random variables are not necessarily independent Exercise Let X and Y be the discrete random variables Whose joint distribution is given in the table on the right Demonstrate the X and Y are uncorrelated7 but dependent Second list of properties 1 Cova bXc dY deovXY CorrXY 2 Corra bXc dY Let a7 b7 0 and d be constants Then when bd gt 0 iCordX Y when bd lt 0 As a general example let X be the height of an individual in feet and Y be the weight of the individual in pounds If we wanted to change the measurement scales so that height was measured in inches and weight in ounces7 then the covariance would change since C3901212X16Y1216C39012XY7 but the correlation would remain the same since Corr12X7 16Y C390MX7 Y 453 Correlations for the Standard Models The following table gives the correlations for the standard bivariate models with parameters pm My am 0y p Probability Model Correlation Bivariate Hypergeometric Distribution C390MX7 Y 7 with parameters 71 M17 M27 M3 Trinomial Distribution ComX Y 7 1571 1572 with parameters 71 p1 192193 Bivariate Normal Distribution C390MX7 Y p 454 Conditional Expectation Regression Discrete case Let X and Y be discrete random variables with joint PDF pm7 1 If pXm 7 07 then the II r I I or the II mean of Y given X z EYlX m is de ned as follows EYlX 95 Z 24 PYXzyl7 yeRw where Ry m is the conditional range the collection of y values with py Xmylm 7 07 provided the series converges absolutely 2 If pyy 7 07 then the II r I I or the II mean of X given Y y is de ned as follows EXlYy Z PXYyly7 meRx y where Rm y is the conditional range the collection of x values with pX yymly 7 07 provided the series converges absolutely Continuous case Let X and Y be continuous random variables with joint PDF fmy 1 If fXm 7 07 then the II r I I or the II mean of Y given X z EYlX m is de ned as follows EmX z y mam dy Rylw where Ry m is the conditional range the collection of y values with fy Xmylz 7 07 provided the integral converges absolutely 2 If fyy 7 07 then the II r I I or the II mean of X given Y y is de ned as follows EltXY y R z hiya90W dm My where Rm y is the conditional range the collection of x values with fX yymly 7 07 provided the integral converges absolutely Regression The formula for the conditional mean EYlX m as a function of m is often called the regression of Y on X Similarly7 the formula for the conditional mean EXlY y7 as a function of y is often called regression of X on Y EIYX Exercise Let X and Y be the discrete random variables y Whose joint distribution is given in the following table Find EYX x for x 012 Exercise Let X and Y be the continuous random Varii ables Whose joint PDF is 1 fzy 1 When Jay E R 0 otherwise7 Where R C R2 is the region Rzy0 z 2124 y 2x24i Find EYX x for 0 g x g 2 E YIX Exercise Let X and Y be the continuous random Varii ables Whose joint PDF is 2 ny 6y7x When0ltacltylt37 and 0 otherwise Find EYX x for 0 lt x lt 3 E YIX Theorem Linear Conditional Expectation Let um EX7 My EY7 am SDX7 0y SDY and p C390MX7 Y If the conditional expectation EYlX z is a linear function of x then the form of this function must be pa EYlX 95 My Ti95 Hz 1 Exercise continued Let X and Y be the continuous random variables with joint PDF given on page 27 Use the work that we did on pages 21 and 27 to demonstrate that the conditional expectation of Y given X z satis es the form given in the theorem above 455 Historical Note Regression To The Mean The term regression was coined by the British scientist Sir Francis Galton in the late 1800 s Galton and his young associate Karl Pearson were interested in studying family resemblances They used the bivariate normal distribution in their studies Bivariate normal distribution The bivariate normal distribution has ve parameters Hz EXv My EYv Um SDX7 0y SDY7 P 0077X7Yb and a complicated joint PDF m A useful way to think about the joint PDF is as the product of a marginal PDF with a conditional PDF as follows 11z7yfxfYLXmyh 7 x alrealpansltzy This is useful because 1 X has a normal distribution with mean pm and standard deviation am and 2 the conditional distribution of Y given X z is normal with parameters Ewmawfy xi7m and SDY Xm 03 1702 m Joint fatherson height distribution In one study Galton and Pearson measured the heights of 1078 fathers and their grown sons and developed a bivariate normal distribution of father son heights in inches with the following parameters EX 677 EY 687 SDX 27 SDY 27 CorrX Y 050 Further the expected height in inches of a grown son whose father is z inches tall is EmX z 687 Wag 7 677 3485 05x for each x Since sons are on average 1 inch taller than fathers and since the standard deviations of the height distributions are equal Galton initially thought that the line y z 1 would represent the conditional expectation Instead he got the line y 3485 05z 1This historical note uses material from Chapter 25 of the textbook written by Freedman Pisani and Purves Norton amp Company 1991 1 Left Plot The left plot shows the GaltonePearson model for the joint fathereson height distribution 2 Rtght Plot The right plot is a plot of the zyeplane centered at malty 677 687 he dashed line is the line L z 1 and the solid line is the line L 3485 051 For each I the Value of g on the line L 3485 051 is midway between the Values on o thelineyz1and o the horizontal line L 687 This lead Galton to observe that the conditional expected Values had regressed toward the mean for all sons Probabilistic basis for regression to the mean Lastly after studying the works of Gree gor Mendel2 and Sir Francis Galton the British statistician R isher proved that Galton7s idea of regression to the mean77 was a consequence of the probabilistic models of inheritance pioneered by Mendel 2Gregor Mendel 182271884 is the Austrianebozn Augustinian m ho is considered to be the father of modern genetics He postulated the existence of entities now called genes and deve oped si p e chanc mode to explain how genotypes the set of genes present in an individual a ect phenotypes the observable physical characteristics of an individual 46 Expected Value of a Linear Function of a Random kTuple In this section7 we consider summarizing linear functions of the form k 9X17X27 7Xk a Zbinw i1 where a and the his are constants7 and X17 X27 Xk is a random k tuple 461 Mean and Variance The mean and variance of a linear function of the Xi s can be written in terms of the means and variances of the Xi s7 and the covariances CovXi7 Xj for i 7 j Theorem Linear Function Let X17 X27 7Xk be a random k tuple with nite means7 variances and covariances7 and let V a 21 biXl be a linear function of the X s7 where a and the his are constant Then 1 EV a 211 bEX 2 VmV 21 bgva 21 bibjCovXi Xj Exercise Suppose that the random triple X7 Y7 Z has the following summaries EX EY EZ 10 and VarX VarY VarZ 4 andletV12X73Y4Z a Find the expected value of V b If X7 Y and Z are mutually independent7 nd the variance of V c If C390MX7 Y C390MX7 Z C390MY7 Z nd the variance of V 462 Covariance The covariance between two linear functions of the Xi s can be written in terms of the means and variances of the X s7 and the covariances CovXl7 Xj for i 7 j Theorem Covariance Let X17 X27 7Xk be a random k tuple with nite means7 vari ances and covariances7 and let k k VaZblXi and WcZdlXi 391 391 l 1 be linear functions of the X s7 where 017 c and the his and dis are constants Then k CovV W Z bidiVaMXi Z bideovXi Xi i1 1739 Exercise Suppose that the random triple X Y Z has the following summaries EX EY EZ 10 and VarX VarY VarZ 4 andletV12X73Y4ZandW2XY7Z a If X Y and Z are mutually independent nd CovV W and CorrV b If C39orMX7 Y C39orMX7 Z COTTY Z nd C390vV7 W and C390MV7 463 Random Sample Sample Sum Sample Mean A random sample of size k from the X distribution is a list X17 X27 Xk of k mutually independent random varialoles7 each with the same distribution as X Given a random sample of size k 1 Sample Sum The sample sum is the quantity S 21 Xi 2 Sample Mean The sample mean is the quantity Y 1 Xi 139 Exercise Let X17 X27 Xk be a random sample from a distribution with mean a EX and standard deviation 0 SDX Find expressions for the mean and variance of the sample sum7 and for the mean and variance of the sample mean 464 Independent Normal Random Variables A linear function of independent normal random variables is itself a normal random variable Theorem Independent Normal RVs Let X17 X27 Xk be independent normal random variables7 and let V a 21 biXi where a and the his are constants Then V is a normal k k random variable with EV a Z biEXl and VarV Z bgvai i1 i1 Exercise The Big Target Corporation is offering special training to its most promising junior salespeople To qualify for the program7 a sales associate must take verbal and quantitative exams7 and earn a combined score C 2 Verbal Score 3 Quantitative Score of 500 points or more Assume that verbal scores can be modeled using a normal distribution with mean 80 and standard deviation 107 quantitative scores can be modeled using a normal distribution with mean 120 and standard deviation 157 and scores are independent Find the probability that a junior salesperson quali es for the special training program 47 Moment Generating Function The moment generating function MGF of a random variable X is the function M t E6tX if the expectation exists where t is a real number Properties The following properties can be proven using properties of expectation and properties of derivatives A Uniqueness lf Mt exists for all t in an open interval containing 0 then it uniquely determines the probability distribution of X B Derivatives lf Mt exists for all t in an open interval containing 0 then Mltkgt0 EXk for k 012 where Mkt is the kth derivative of the moment generating function 0 Linear Function Let MXt be the MGF of X Myt be the moment generating function of Y and suppose that Y a bX Then Mm eatMXbt for all t The moment generating function of a linear function of X has a speci c relationship to the moment generating function of X 5 Independence If X and Y are independent random variables and W X Y Mwt MxtMyt for all t The moment generating function of the sum of independent random variables is the product of the moment generating functions Note Suppose that Mt exists for all t in an open interval containing 0 Then we know from Property B that the Maclaurin series expansion of Mt must be MWO k EXk k AMP k ti k t 0 0 and we know from Property A that Mt uniquely de nes the distribution Thus the sequence of expected values EXk for k 012 known as the sequence of kth moments must de ne the distribution uniquely Exercise Let X be an exponential random variable with parameter 1 Use the moment generating function of X to nd EXk for k 1 2 3 4 Exercise Let X be a normal random variable with mean 0 and standard deviation 1 Use the moment generating function of X to nd EXk for k 1 23 4 Exercise a Demonstrate that the moment generating function of a Poisson random variable with parameter is Mt e Vgt 1 Let X be a Poisson random variable with parameter 57 let Y be a Poisson random variable with parameter 87 and let W X Y If X and Y are independent7 use moment generating functions to demonstrate that W is a Poisson random variable with parameter 13 Exercise Use the de nition of the moment generating function to verify Properties C and D Exercise Let X be a normal random variable with mean u and standard deviation 0 and let Z be the standard normal random variable Use the facts that XuaZ and Mzt 5 22 to nd a general form for the moment generating function of a normal random variable Exercise Let X be a normal random variable with mean um and standard deviation am Y be a normal random variable with mean My and standard deviation Ty and assume that X and Y are independent Use moment generating functions to demonstrate that W XY is a normal random variable Exercise Let X be a normal random variable with mean um and standard deviation am Y be a normal random variable with mean My and standard deviation Ty and assume that X and Y are independent Use moment generating functions to demonstrate that W a bX cY is a normal random variable7 where 017 b and c are constants

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "I bought an awesome study guide, which helped me get an A in my Math 34B class this quarter!"

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.