Statistical Signal Processing I
Statistical Signal Processing I EE 262
Popular in Course
MTED 5322, Pedagogical Content Knowledge in Teach Math
verified elite notetaker
verified elite notetaker
verified elite notetaker
CRJU 311 003
verified elite notetaker
Popular in Electrical Engineering
This 5 page Class Notes was uploaded by Amiya Haley on Monday September 7, 2015. The Class Notes belongs to EE 262 at University of California - Santa Cruz taught by Staff in Fall. Since its upload, it has received 78 views. For similar materials see /class/182337/ee-262-university-of-california-santa-cruz in Electrical Engineering at University of California - Santa Cruz.
Reviews for Statistical Signal Processing I
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 09/07/15
Julio L Peixoto September 1999 INDEPENDENCE AND UNCORRELATION Some random comments motivated by class discussion Statistical Independence l N E 4 Random variables y1y2yn are saidtobe quot quot quotJ39 J l J if39K 39 J about one or several of them does not affect the probability distribution of the others The typical model is that of tossing a coin repeatedly The results of one or several tosses does not affect the probabilities of results in the other tosses The coin has no memory it does not know what happened before Assuming their probability density functions pdf s exist y1 y2 yquot are independent if and only if the joint pdf is identical to the product of the corresponding marginal pdf39s By quotidenticalquot we mean quotequal for all values these random variables can takequot More formally using fairly standard notation and assuming the pdf39s eXist y1 y2 yn are independent if and only if fyhypuw puzw au 131u1fyzuzm nun for all u1u2un where u denotes any value y can take In the case of discrete random variables the densities become probabilities and it is convenient to change the fs to p s as I ll do in Examples 1 and 2 below Another interesting characterization is this one y1 y2 y if n are independent if and only Eg1y1g2y2gy Eg1y1Eg2 y2Egy for all functions g1r g2 r g r Actually it should say for all measurable functions but I don t want get into that type of detail I couldn39t use fs for the functions because I had already usedf39s for the pdf39s Hence the g39s Uncorrelation 5 6 Uncorrelation is sometimes confused with independence but it is a different concept Correlation is a measure of linear dependence as we shall see on this course Review of basic concepts I Variance ofy Vary Ey2 EV2 Ey2 a measure of the variability of y I Covariance between y1 and y2 Covyy2 In Eylly2 EylEyy2EylEy2 measure of the linear covariation between y1 and y2 C0Vy1y2 Vary1 Vary2 of the linear relationship between y1 and y2 A correlation is always between 1 and 1 I Correlation between y1 and yz r yl y a measure of the strength JLP Independence and uncorrelation Page 2 Relationship between Independence and Uncorrelation 7 9 By comments 4 and 6 above y1 and y2 are independent if and only if any function of y1 is uncorrelated with any function of y2 For example independence of y1 and y2 requires not only the uncorrelation between y1 and y2 but also the uncorrelation between yl2 and y2 between logy1 and eyZ between ly1 and sing2 between any function of y1 and any function of y2 Clearly independence implies uncorrelation but the converse is not true Independence is a much stronger requirement However uncorrelation and independence are equivalent if we have normal distributions More precisely if two variables are jointly normally distributed meaning that any linear combination of the two variables is normally distributed then these two variables are independent if and only if they are uncorrelated This is an exceptional property of the normal distribution which cannot be extended to other distributions Example 1 Uncorrelated but not Independent 9 Assume we have the four data points in the following graph each with the same probability 025 Obviously y1 and y2 are not independent since y2 ylz R 2 1 1 2 yl 10 The joint probability distribution of y1 and y2 is given on the following table yl l 2 l 2 2 1mm mm 025 050 025 025 025 025 100 11 Note that the joint inside probabilities are not the products of the marginal probabilities a sign of dependence Read comments 2 and 3 12 Another sign of dependence If we know that y1 is 2 then the probability that y2 is 4 is l ie Py2 4 y1 2 l ifwe know that y1 is 1 then the probability that y2 is 4 JLP Independence and uncorrelation Page 3 is 0 ie P022 4 y1 l 0 Hence knowledge of y1 affects the probability distribution of y2 We have in fact complete dependence here if we know y1 we know y2 with complete certainty l3 Calculation of some expectations Recall that Egygy2 Z yz 8111gzy21vyyz ypyz where fm2 rgt pmZ gtr is the joint probability distribution function of y1 and y2 To save space I use pto denote pmZ y1y2 in the following table y y P MD yzp ylyz p yf p yf yz p 111y2 p yf 1n yz p 2 4 025 050 100 200 100 400 03466 13864 1 1 025 025 025 025 025 025 02500 02500 1 1 025 025 025 025 025 025 02500 02500 2 4 025 050 100 200 100 400 03466 13864 100 000 250 000 250 850 11932 32728 14 Are y1 and y2 uncorrelated Covyy2Eyy EyEy000 0002500 Yes they are uncorrelated If we fit a leastsquares line on the above graph we get a horizontal line Hence y1 and y2 are uncorrelated even if they are not independent 15 Are yl2 and y2 uncorrelated Cov 12y2 ELnyZ EV12Ey2 850 250250 225 No they are correlated ie not uncorrelated In fact the correlation between yl2 and y2 is 1 since y2 y12 Since y1 and y2 are not independent there exist functions of y1 and y2 that are correlated ie not uncorrelated See comment 4 above ON Are yl2 and lny2 uncorrelated Cov 12lny2 E012 lny2 Eyf Elny2 32728 25011932 02898 No they are correlated ie not uncorrelated Read previous comment JLP Independence and uncorrelation Page 4 Example 2 Independent and hence Uncorrelated 17 Assume we have the four data points in the following graph each with the same probability 025 A O 4 O O 1 O 1 1 yl 18 The joint probability distribution of y1 and y2 is given on the following table yl 1 1 y 1 025 025 050 4 025 050 050 050 100 19 Note that the joint inside probabilities are the products of the marginal probabilities Hence y1 and y2 are independent Read comments 2 and 3 20 Calculation of some expectations y1 y2 P ylp yzp ylyzp yf p yfy2 p 11102 P yf 1n yz p 1 4 025 025 100 100 025 100 03466 03466 1 1 025 025 025 025 025 025 02500 02500 1 1 025 025 025 025 025 025 02500 02500 1 4 025 025 100 100 025 100 03466 03466 100 000 250 000 100 250 11932 11932 21 Are y1 and y2 uncorrelated Covyy2 E00 Ey Ey 000 000250 0 Yes they are uncorrelated 22 Are yl2 and y2 uncorrelated Cov 12y2 Eny2 Ey12 Ey2 250 100250 000 Yes they are uncorrelated JLP Independence and uncorrelation 23 24 Page 5 Are yl2 and lny2 uncorrelated Cov 12lny2 E012 h1y2 EVIZElnyZ 11932 10011932 00000 Yes they are uncorrelated Since y1 and y2 are independent any function of y1 is uncorrelated with any function of y2 Read comment 4 above Final Remarks 25 26 I hope this has helped to clear the confusion between independence and uncorrelation Independence means that knowledge of one variable does not tell us anything about the probability distribution of the others Think of tossing coins rolling dice playing the lottery all processes with no memory ie independent Correlation is a measure of the strength of the linear relationship Uncorrelation ie zero correlation zero covariance means that the leastsquares line is horizontal Independence implies uncorrelation The converse is not true as Example 1 shows Uncorrelation does not imply independence However in the case of normal variables uncorrelation does imply independence More precisely two jointly normal variables are independent if and only if they are uncorrelated The following Venn diagram may help you understand what is possible and what is not inde endent jointly normal 390 A V uncorrelated The above diagram defines 8 classes of pairs of random variables However classes 3 5 and 7 are empty
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'