New User Special Price Expires in

Let's log you in.

Sign in with Facebook


Don't have a StudySoup account? Create one here!


Create a StudySoup account

Be part of our community, it's free to join!

Sign up with Facebook


Create your account
By creating an account you agree to StudySoup's terms and conditions and privacy policy

Already have a StudySoup account? Login here

Statistical Theory I

by: Jordane Kemmer

Statistical Theory I ST 521

Jordane Kemmer
GPA 3.79

Hao Zhang

Almost Ready


These notes were just uploaded, and will be ready to view shortly.

Purchase these notes here, or revisit this page.

Either way, we'll remind you when they're ready :)

Preview These Notes for FREE

Get a free preview of these Notes, just enter your email below.

Unlock Preview
Unlock Preview

Preview these materials now for free

Why put in your email? Get access to more of this material and other relevant free materials for your school

View Preview

About this Document

Hao Zhang
Class Notes
25 ?




Popular in Course

Popular in Statistics

This 30 page Class Notes was uploaded by Jordane Kemmer on Thursday October 15, 2015. The Class Notes belongs to ST 521 at North Carolina State University taught by Hao Zhang in Fall. Since its upload, it has received 23 views. For similar materials see /class/223948/st-521-north-carolina-state-university in Statistics at North Carolina State University.

Similar to ST 521 at NCS


Reviews for Statistical Theory I


Report this Material


What is Karma?


Karma is the currency of StudySoup.

You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/15/15
Chapter 4 Multiple Random Variables We study the joint distribution of more than two random variables called a random vector such that XY X Y Z X1 Xn and the distri bution of their functions like X Y XYZ or X1 X2 X 1 Bivariate Random Variables Assume both X and Y are random We treat X Y as a two dimensional random vector and study their relationship 11 Discrete Case Assume that both X and Y are discrete random variables with the sample space X and 3 respectively Joint pmf fxyy PX mY y Vm E Xy E 3 Properties fXY957y Z 0 39 EweY Zyey fXY907y 1 The probability of a set A is given by PXY6A Z fXY7y 41 EA Marginal pmf If the joint distribution of X Y is known their marginal pmf are may PltX z Z away 263 My PltY y Z away 16X 74 EXAMPLE 1 Two fair dice thrown Let Xmaximum Ysum Possible values X 1 2 3 4 5 6 Y 2 3 4 5 6 7 8 9 10 11 12 Can write the probabilities in a table REMARK 0 Joint distribution determines the marginal distribution 0 Marginals do not determine the joint distribution EXAMPLE De ne the joint pmf by 101 0 no 1 2110 2111 may 0 otherwise Consider another joint pmf by 1 5 3 i 00 A E 10 A 3 no 1 A flt11 A E my 7 001herw1se They share the same marginal distributions but not the same joint distri butionl 75 12 Continuous Case Assume that both X and Y are continuous random variables Joint pdf A function fxy7 y is called ajoz39ntprobabz39lz39ty density function of X Y if PXY e A fmydzdy VA 6R2 zy6A The joint pdf satis es WW 2 0 ff ff fxyzydmdy 1 Joint cdf The joint distribution of X7Y can be completely described With their joint cdf Haw PX m7Y y7 Wang 632 Relationship between joint pdf and joint cdf y m Fxy f 7 fXJu12luo lv7 82 mFWQ Hm Marginal pdf If the joint pdf of X7 Y is known7 their marginal pdfs are given by 00 mm m my 700 My mwdz 76 EXAMPLE Check whether the following function a valid pdf ay ye y10 lt m lt y EXAMPLE fzy 2I0 g x g y g 1 77 EXAMPLE fzy e yI0 lt z lt y 78 13 Expectation of Functions of Random Vector Assume g is a real valued function of two random variables gX Y If X and Y are both discrete then E9X7Y Z Zg7yfXY7y39 16X yell If X and Y are both continuous then 00 00 mey gownyWme 700 700 Properties 0 EaX bY c aEX bEY c ummxnmmxwawmmywwmmxwwa o In general EXY 7 EXEY unless X and Y are independent M Mxyt s EetXsY 79 EX e yI0 lt m lt Compute EX7EY7EXY7Mxyt7 s 80 2 Conditional Distributions 21 Discrete Case Assume both X and Y are discrete For any x such that PX z gt 07 the conditional pmf of Y given X z is de ned as PXz7Yy PltXm Vyey leiWWW PW y X 95 We can de ne fX ymy similarly REMARK The function is indeed a pnif7 since for any xed x it satis es 39 fYXy Z 0 for any y 2 waMQE 1 EXAMPLE The two dice example7 Xn1axin1un17 Ysun1 fYXy 339 fXY7 81 22 Continuous Case Assume both X and Y are continuous For any x such that fX gt 07 the conditional pdf of Y given X z is de ned as 7 WW V fYXy 7 baggy y 6 3 We can de ne fX ymy similarly REMARK The function fy Xyz is indeed a pdf7 since for any xed x it satis es fYXy Z 0 for any y 0 L fYXydy 1 EXAMPLE Assume fzy e yIO lt z lt Compute fy Xym 82 23 Conditional Mean and Variance For discrete random variables EYlX 95 nywXltylgt7 y VarYX ZyEYX2fyixy For continuous random variables EmX z yfy XWWda VarmX z y 7 max 2fyixylmdy REMARK As before7 we have VarmX z EY2X z 7 EmX 35 EXAMPLE Two dice example7 Xmax7 Ysum Compute EYX 3 EX e yI0 lt m lt Find EY X m and VarY X 83 Remark Note EYlX z is a function of m Therefore7 EYlX is a random variable as a function of X E9XlX 9X Theorem 0 Conditional Expectation ldentity EY o Conditional Variance ldentity VarY EVarYlX VarEYlX REMARK E9X7Y EE9X7YlY EE9X7 WW 0 Conditional expectation as projection EY 7 EYlX2 EY 7 gX2 v 9 function So EYlX is closest in above sense to Y among all the functions of X 84 3 Independence Def Let X7 Y be a bivariate random vector withjoint pdfpmf gmm7 y7 Then X and Y are called independent random variables if for every m y E R aw fxfyy EXAMPLE Consider the discrete bivariate random vector X7 Y with joint pmf given by H1071 K2071 H20 2 f1072 f1073 2073 Are X and Y are independent Lemma Let X7Y be a bivariate random vector with joint pdf or pmf fXJmy7 Then X and Y are independent random variables if and only if there exist functions gz and My such that for every x E R and y E R7 Wm 24 9hy In other words7 the joint pmfpdf is factorizable We do not need to com pute marginal pdfs EXAMPLE Consider the continuous bivariate random vector X7Y with joint pdf given by 1 H9579 952945 yiw2 90 gt 07 y gt 0 Are X and Y are independent 85 Theorem If X and Y are independent then i EYlX ii The events X E A and Y E B are independent PX e AY e B PX APY e B VA C RB C R iii 7 E9XhY E9XEhY In particular EXY iv In addition we have Mxya s EetXSY MXtMys And MXyt EetXY MXtMyt If it is easy to identify the right hand side as the MGF of some standard distribution then the sum of two independent variables is easy to nd Example 1 binomialbinomial Example 2 P0iss0nPoiss0n 86 Example 3 negative binomialnegative binomial Example 4 normaln0rmal Example 5 gamma gamma 87 4 Bivariate transformation In this section we only consider continuous bivariate random vector X Y Consider the following bivariate transformation of X Y U 91X7Y7 V 92X7Y7 where 91 and 92 are continuous differentiable and onetoone Therefore we can de ne their inverse transformations as Xh mm Ymavy is the Jacobian determinant or simply Def Jacobian matrix and determinant 7 3 31 Sl l is the Jacobian matrix and detJ the Jacobian Example 1 Linear transform UXK VX7Y Example 2 Polar transform r VXZ Y2 t9 arctan 88 Theorem lf fxy7y is the joint density of XY7 then fUVU7U fXYh1U7 11 h2U7Ul detUN Proof follows from change of variable rules for integration 7 omitted EXAMPLE Sum and difference of independent normals EXAMPLE Polar transform of independent normals 89 EXAMPLE Sum and ratio of independent gammas EXAMPLE Ratio of two standard normals ManytoOne Transformation Assume X7Y takes value from A A0 U A1 U U Ak where PXY 6 A0 0 Also U gliXYV gglX7 Y is oneto one transformation from Ai to B7 for i 17 7 k Then k fUVu u Z fXyh1iuu IlaWm detJZl i1 90 5 Hierarchical Mixtures Recall that EY EEYX7 varY Emmip0 varEY X EXAMPLE binomial Poisson An insect lays a large number of eggs7 each surviving with probability p On the average7 how many eggs will survive Let X the number of eggs that survive Let Ythe total number of eggs laid by the insect 1 Describe their distributions 2 Find the joint distribution of X7 Y 3 Find the marginal distribution of X7 EX and VarX 91 EXAMPLE Poisson gamma EXAMPLE chi squaregamma 92 EXAMPLE binomial beta EXAMPLE binomial Poisson gamma optional 93 6 Covariance and Correlation Covariance A measure of joint variation CovXY EX 7 EXY 7 Note the outer expectation is with respect to the joint distribution of XY And we have CovX Y EXY 7 Correlation 7 CovXY pXY UXUY REMARK If X and Y are independent then covX Y 0 and pr 0 But the converse is not true EXAMPLE X N N0 1 Y X2 EXAMPLE X X1 X3 Y X2 X3 where X1X2X3 pairwise inde pendent with common variance 02 Compute p 94 EXAMPLE X Unif017 Z Unif0T1 and they are independent Let Y X Z Compute p EXAMPLE Y X2 Z7 where X7 Z are independent7 X symmetric about 07 Z any distribution Compute CovXY VaraX bY aZVarX bZVarY 2abCovX Y CauchySchwarz Inequality lCovXYl aXay with equality iff X and Y are linearly related Corollary 71 pXY S 1 And lpxyl1iffYaXbwp17whereagt0ifpry1andalt0iff PXY 1 Proofs can be found in the textbook and are omitted here 95 7 Bivariate Normal we say Xx BVNltp1p2a a pgt if 1 z f y 27139x17p203910392 imltgt32pltgtlt gtltgtZH We will show that X N Nma Y N wag pr p aX bY is normal 96 Conditional distribution for Bivariate normal Suppose X7Y N BVNM1p2afa p then 01 XlYy NltM1P 72Wil 2gt7 0mm 02 YXz NltM2pajm7 03mm 8 Multivariate Distributions Several variables X17 7Xn 81 Discrete Case Joint pmf fX1VVVXnm1mn PX1 1 Xn satiSfyng fX1mXn9017 7 Z 0 and 2mm fX1mXn177n1 For any subset A of R 7 we have PX1XneA Z fX1VVVXnz1zn mbm meA Marginal distribution of Xi17 7Xlk inleQ Mun7 Z fX1vvvXn177n39 other indices ln particular7 one dimensional marginals mm Z fm1mn m1gtvgtmi71gtmi1gtmgtmn Conditional distribution fx1xn ka1XnX1mXkk17 7 nl17 3971Ui2 Covariance CovXl7 Xj 7 based on pairwise distribution 97 Independence X1 Xn are called mutually independent random vari ables if their joint is the product of marginals fX1Xn17 7 90m H ini7 V9017 7 i1 REMARK lf X1 Xn are mutually independent then 1 Any pair X and Xj are pairwise independent 2 Functions 91X1 gnXn are independent and n HE9iXi39 i1 7L EH 9109 i1 3 MGF is the product of indiVidual MGF s 7L MXIXnt1 tn i1 4 Let Z X1 X then the mgf of Z is 71 MN HMXt i1 In particular if X1 Xn all have the same distribution with mgf MXt then M205 lMxtl Applications i Sum of independent normals is normal Mean variance add up ii Sum of independent gammas with the same scale parameter is gamma with the same scale and shape parameter added up In particular sum of independent exponentials is gamma iii Sum of independent Poisson is Poisson with parameters added up iV Sum of independent geometric is negative binomial 98 Multinomial distribution 71 categories7 and each item can be from one and only one category Sampling m times independently from the categories with probabilities pl 719 where p1 pn 1 Let X the count of the ith category Let m1 Wm be nonnegative integers adding up to m Then P X i X i i m m1 M M 1 7m17n7zn7mpl p2 mpn39 Prob add up to one7 as they are the terms in expansion of p1 pnm i marginals are lower order multinomial One dimensional X Binmpi ii Conditionals iii Merging X1X2X37 7 X N Multinomialmp1p2p37pn iv VarXl nal1719i and CovXl7 Xj impipj for all i 7 j 99 82 Continuous Case 10th pdff17 wmn fX1XT9017 WM satiS eS fX1Xn17 7 Z 0 and fleVwXTle mndz1 dzn 1 Probabilities are obtained by PX17 7Xn E A fX1Xn177nd1 dn 11mn6A wof XMwa fX1XT17 7 er 90 9 21 2k 17 7 k other indices ln particular7 one dimensional marginals in f9017 790001951 di71di1 39 39 39dg n Conditional fx1xn ka1anX1Xkk17 7 nl17 3971Ui2 Covariance covXi7 Xj 7 based on pairwise distribution Independence joint is the product of marginals Equivalently7 MGF is the product of individual MGF s EXAMPLE 1 Uniform over the ball 3 fx1m2m3 1 mg m lt1 100 EXAMPLE 2 Dirichlet 9017 7171 F0 1 0 16 04171 akilil akil j 0 1 Plta1fak 951 95hr 9 9 gt 7951 t t 90k Properties i marginals are lower order Dirichlet One dimensionals are beta ii Conditionals iii Merging of categories iv Covariances 101 EXAMPLE 3 Multivariate normal Joint density of Y Y17 7Y fYyl7 7y W exp WTEAW Ml Generation of multivariate normal NW 2 1Let X1 Xn be iid N0 1 2 Write X X17 Xn and let Y AX In where E AAT ie the Cholesky decomposition Then Y is multivariate normal with the above density function7 with EY u and VarY E Marginals and conditionals are also multivariate normal 102 9 Some Useful Inequalities A CAUCHY SCHWARZ EXY2 EX2EY2 B HoLDER EXY S EX 1 EXV WR where p 1 q 1 1 C MINKOWSKI EHX W S EX 1 EY 1 7 for p 2 1 D JENSEN A function 1 is called convex if zMat 17 13 11105 17 a11s For any convex function 1 E X Z wEX 103


Buy Material

Are you sure you want to buy this material for

25 Karma

Buy Material

BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.


You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

Why people love StudySoup

Steve Martinelli UC Los Angeles

"There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

Allison Fischer University of Alabama

"I signed up to be an Elite Notetaker with 2 of my sorority sisters this semester. We just posted our notes weekly and were each making over $600 per month. I LOVE StudySoup!"

Jim McGreen Ohio University

"Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."


"Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!

Refund Policy


All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email


StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here:

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.