×

### Let's log you in.

or

Don't have a StudySoup account? Create one here!

×

or

16

0

37

# Review Sheet for EMSE 171 with Professor Dorp at GW (3)

Marketplace > George Washington University > Review Sheet for EMSE 171 with Professor Dorp at GW 3

No professor available

These notes were just uploaded, and will be ready to view shortly.

Either way, we'll remind you when they're ready :)

Get a free preview of these Notes, just enter your email below.

×
Unlock Preview

COURSE
PROF.
No professor available
TYPE
Class Notes
PAGES
37
WORDS
KARMA
25 ?

## Popular in Department

This 37 page Class Notes was uploaded by an elite notetaker on Saturday February 7, 2015. The Class Notes belongs to a course at George Washington University taught by a professor in Fall. Since its upload, it has received 16 views.

×

## Reviews for Review Sheet for EMSE 171 with Professor Dorp at GW (3)

×

×

### What is Karma?

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 02/07/15
EMSE 171271 DATA ANALYSIS For Engineers and Scientists Session 5 Vectors and Matrices Matrix Algebra Linear Combinations Coordinate Systems Geometric Interpretation THE GE 0 RG E WASHINGTON UNIVERSITY WASHINGTON DC Lecture Notes by J Ren van Dorp1 wwwseasgwuedudorpjr 1 Department of Engineering Management and Systems Egineering School of Engineering and Applied Science The George Washington University 1776 G Street NW Suite 110 Washington DC 20052 Email dorpjrgwuedu STATISTICAL REVIEW Vectors and Matrices 0 Typical convention is that vectors are written as columns The i th element of a vector is indicated by 1 Hence an n dimensional vector is 0 Convention Underline to indicate a vector or write them in a bold font 0 An m X n matriX A may be viewed as n columns each of dimension m lts elements are indicated by aij where the indeX i refers to the row number and the indeX refers to the column number G11 G12 171 A 921 a aml u amn EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 122 STATISTICAL REVIEW Matrix Algebra 0 Convention Use capital letters for matrices and often they are written in a bold font 0 If m n then the matrix is called a square matrix 0 If aij aji for all elements of a square matrix then the matrix is called symmetric 1 5 6 7 5 2 8 9 A 6 8 3 10 7 9 10 4 0 If aij 0 for all off diagonal elements of a square matrix then the matrix is called a diagonal matrix 39 If aii l for all on diagonal elements of a diagonal matrix then the matrix is called the identity matrix and is usually denoted by I 0 An n dimensional vector is an n X 1 matrix EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 123 STATISTICAL REVIEW Matrix Algebra VectorScalar multiplication 1 A1101 1 2 2 A162 2 4 Ag A egg 2 3 6 lb Man 4 8 Conventions Write 11167 or g to indicate a transposed vector A transposed column vector becomes a row vector and vice versa 0 An m X n matriX may also be viewed also as m row vectors each of dimension n EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 124 STATISTICAL REVIEW Matrix Algebra MatrixVector multiplication n Eat1 G11 G12 am j1 G21 G22 G2 301 n n 1102 Zazjivj Ag 39 j1 am 1n I mm H aml amn 1 amn Zam j j1 m X n matrix n vector Tn vector m X n matriX n X 1 matriX m X 1 matriX Example 122334 20 224354 36 goon EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 125 STATISTICAL REVIEW Matrix Algebra VectorMatrix multiplication G11 G12 G17 G21 G22 G27 gTA 1 1102 mm 39 39 am 1n aml amn 1 amn m m m Email ZZUZ39CLZQ i1 i1 i1 Tn vector m X n matrix TL vector 1 X m matrix m X n matrix 1 X n matrix lt4 5 i a 4152 4254 435514 28 37 Example EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 126 STATISTICAL REVIEW Matrix Algebra MatrixMatrix multiplication G11 G12 a1 Z711 Z712 7117 G21 G22 G27 Z721 Z722 52p AB am 1n bn 1p aml amn 1 amn bnl bnp 1 bnp Zaljb Zaljb Zaljbjp j1 i1 j1 n n n Eda521 Eda522 Za2jbjp j1 j1 j1 n E am 1jbjp j1 n n n Zamjb Zamjbm l Zamjbjp m X n matrix n X p matrix m X p matriX EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 127 STATISTICAL REVIEW Matrix Algebra 123 245 5 112335 122436 22 28 214355 224456 39 50 Example HgtN 0 MatrixMatrix multiplication is NonCommutative First of all we have to consider square matrices in this case why But even when we consider square matrices the following is not true in general AB7 BA Alt gtBlt 2 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 128 Example STATISTICAL REVIEW Matrix Algebra AB 1 2 1 1122 1324 5 11 3 4 2 3142 3344 11 25 BA 1 3 1 1133 1234 10 14 2 4 3 2143 2244 14 20 Transpose of a matrix G11 012 am G11 G21 aml G21 G22 G27 012 G22 am2 A 39 3 AT i 3 am 1n amn 1 aml amn 1 amn aln am 1n amn EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 129 STATISTICAL REVIEW Matrix Algebra Example 1 4 A gtAT 2 5 3 6 m X n matrixT n X m matrix Transpose of a matrix product ABT BTAT m X n matrix n X p matrixT n X p matrixT m X n matrixT p X n matrix n X m matrixT p X m matrix EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 130 STATISTICAL REVIEW Matrix Algebra Example 2 4 6 i i 44 56gt 8 10 12 5 6 98 128 T T 2 4 6 2 i ii 2 4 6gtT 8 10 12 5 6 5 6 8 10 12 135 i18044984456T 246 612 5612898128 The inverse of a square matrix is defined such that A1A AA1 I where I is the identity matrix EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 131 STATISTICAL REVIEW Matrix Algebra Example 0 The inverse of a matrix product AB 1 B 1A1 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 132 STATISTICAL REVIEW Matrix Algebra Example EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 133 STATISTICAL REVIEW Linear Combinations De nition Given a collection of random variables X1 Xn and n numerical constants a1 an the rv Y a1X1l 01an i1 is called a linear combination of the Xi39s Hence we can identify two vectors QT a1 an CT X1 X and write Y QT 1 X 1 matriX 1 X n matriX n X 1 matriX LetECgwheregTJl Mnthen EQT EY am awn fa QTEX Recall E aX CLE X EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 134 STATISTICAL REVIEW Linear Combinations If the Xi39s are mutually dependent vm ZZaECOVXZXj Note COVXX VX i1 j1 Introducing the variance covariance matrix of C COUltX1X2 COUltX1Xn Z COUltX2X1 CovXnX1 39 m we derive VY QTZQ The variance covariance matrix is a square symmetric matrix why The variance covariance matrix is positive definite ie QTZQ gt 0 for all vectors g 75 Q Recall VX gt 0 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 135 STATISTICAL REVIEW Coordinate Systems For a coordinate system one needs three things n yaxis 01 I I 110 I 1 0 xaxis 01 1 An origin 0 0 2 Two lines called coordinate axes that go through the origin In the system above each line is perpendicular which makes it a Cartesian System 3 One point other than the origin on each aXis to establish scale These points identify the standard base vectors g 1 0 and g 0 1 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 136 STATISTICAL REVIEW Coordinate Systems Having a coordinate system allows us to assign to each point in the system its coordinates a1 a2 A yaxis Ia11 a2 b2 A A V 0 We may also write a1 a2 as a vector Note that 2ltgta1ltgta2lt2gt EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 137 STATISTICAL REVIEW Geometric Interpretation Heights and Weights of 20 Women X1 X2 Xd1 Xd2 X51 X82 57 93 585 3060 177427 196516 58 110 485 1360 147098 087341 X1 Height 60 99 285 2460 086439 157984 X2 Weight 59 111 385 1260 116768 080918 Xd1 Height mean centered 61 115 185 860 056109 055230 Xdz Weight Mean Centered 60 122 285 160 086439 010275 X51 Height Standardized 62 110 085 1360 025780 087341 X52 Weight Standardized 61 116 185 760 056109 048808 62 122 085 160 025780 010275 X1 X2 63 128 015 440 004549 028257 Mean 6285 12360 62 134 085 1040 025780 066790 St Dev 330 1557 64 117 115 660 034879 042386 63 123 015 060 004549 003853 65 129 215 540 065208 034679 64 135 115 1140 034879 073212 66 128 315 440 095538 028257 67 135 415 1140 125867 073212 66 148 315 2440 095538 156699 68 142 515 1840 156197 118167 69 155 615 3140 186526 201654 EMSE 171271 FALL 2005 JR van Dorp 1012O5 dorpjrgwuedu Page 138 STATISTICAL REVIEW Geometric Interpretation We can now create two column vectors 51 and 52 that take the values of the standardized variables for height and weight 0 Together these two column vectors form a 20 X 2 matrix given by X s 51 52 0 Each row of these matrix corresponds to one object woman measured on each of two different characteristics height weight By displaying all points in the same coordinate system one can clearly visualize the pattern of observations and the position of each point relative to one another This type of representation is known as a scatter plot EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 139 STATISTICAL REVIEW Geometric Interpretation Scatter Plot of Height and Weight of 20 Women 30 20 39 10 o o E o I II lI I I i 40 39 39 20 30 30 20 10 00 10 20 30 Height Conclusion EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 140 STATISTICAL REVIEW Geometric Interpretation Weight and Height individually tell us less than the two combined Sometimes one would like to have a single heightweightindex describing a person39s characteristics For example Z1 wl xsl wzxsz 30 30 20 10 00 10 20 30 Height Weights 101 and wz can be represented as a vector From the scatter plots the choice QT 101102 1 1 seems to make sense EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 141 STATISTICAL REVIEW Geometric Interpretation VectorScalar multiplication A y39axis T Length of a Vector 1 1 12 a in 2w i1 07l07 xexis wIIWIIZI 1 1N5 x 0707 1 1M 0707 0 The effect of vector scalar multiplication is stretching or shrinking the length of a vector While maintaining its direction EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 142 STATISTICAL REVIEW Geometric Interpretation VectorVector multiplication 2Q 2T llallll llcoslt9gt lt The distance from the origin to the perpendicular projection of the point g a1 a2 onto the line 8 anned b the vector Q e uals QTQ P 57 q EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 143 STATISTICAL REVIEW Geometric Interpretation 0 Consider the following data point from our sample a woman 5 feet tall and weighing 122 pounds Mean 6285 12360 St Dev 330 1557 60 6285 N 122 12360 N W N 086 W N 010 0 Hence this woman is below average height x15 086 standard deviations below the mean and weight x25 010 standard deviations below the mean compared to the other women in the sample What is the women39s height weight indeX T 086 N w as 0707 0707 010gt N 068 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 144 STATISTICAL REVIEW Geometric Interpretation 30 20 TO7O7O7O7 10 00 Weight t o 39 o 39 30 20 10 00 10 20 30 Height 0 A woman of average height and average weight obtains a height weight index of 0 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 145 STATISTICAL REVIEW Geometric Interpretation MatrixVector multiplication We could perform this vector multiplication for every observation in the sample 1XSQ 20x1 20x22gtlt1 30 20 10 00 Weight 10 30 20 10 00 10 20 30 Height EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 146 STATISTICAL REVIEW Geometric Interpretation The vector g1 is a linear combination of the columns of the matrix X 5 X5 51 52 1 X52 wiasi w2 52 This is the same thing we do in multiple regression analysis when we multiply the matrix X containing the values of the explanatory variables in columns by the vector of least squares coefficients Q to obtain the vector of fitted values These fitted values Q are next compared to observed values 31 to conclude whether an adequate model fit has been obtained Model fit will be described by RZ values adjusted RZ values and a normal probability plot of the difference vector y Given an adequate model fit hypothesis test can be formulated regarding the values of the weights wl and 712 These hypothesis use the t distribution and the F distribution EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 147 STATISTICAL REVIEW Geometric Interpretation 0 We could now introduce a new random variable Z 1 describing a random height weight index that is a linear combination of the random variables X15 standardized height and X 2 5 standardized weight where for the given data set Z1 takes the values g1 X81 X82 z1 1 177427 196516 264418 New Varmble Z1 descrlbes 2 2 147098 o87341 165773 persons height weight index 3 39086439 39157984 391 72833 Persons with a high value are 4 116768 o8o918 139786 H d h d h 1 5 056109 055230 078729 m an eaVYgt an W 2 OW 6 086439 o1oz75 068387 value are small and light 7 o2578o o87341 679988 8 o561o9 o48808 o74188 1 353 512 f 337 To describe whether a person s 3 5534 33333 333 height weight combination is I 52333 5252 52197333 above or below that of the average 17 125867 073212 140770 person with the same height 18 095538 1 56699 1 78358 19 156197 118167 194004 welght deX we may mtrOduce 2 20 186526 201654 274485 second variable Z2 which we shall St Dev 1 1 1366409 refer to as bodyweightindex EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 148 STATISTICAL REVIEW Geometric Interpretation MatrixMatrix multiplication A reasonable choice for the body weight indeX Zz is to search for a linear combination of 51 and 52 that is orthogonal to the height weight indeX Z1 Hence we are comparing with one another those persons which have the same height weight index the same value of 21 Informally orthoganality implies that information about Zz does not provide any information about Z1 which implies that Zl and Zz will be uncorrelated 0 Two vectors g and Q are orthogonal if and only if QTQ 0 Example MT 0707 0707 To find the second vector y w12 1022 requires that we solve the equation I 0 0 2 lt 32537 35227 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 149 STATISTICAL REVIEW Geometric Interpretation 22 w12 xs1 szxsz 30 20 10 MT07070707 1TO7070707 Weight 0 O 39 O O O o 30 20 10 00 10 20 30 Height What happens when we multiply the matrix X g 51 52 with the matrix W Q1 Q2 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 150 STATISTICAL REVIEW Geometric Interpretation X51 X52 Z1 22 4 774 4965 2644 0135 4471 0873 4658 0423 0864 4580 4728 0506 4 168 0809 4398 0253 0561 0552 0787 0006 0864 0103 0684 0539 0258 0873 0800 0435 0561 0488 0742 0052 0258 0103 0255 0110 0045 0283 0232 0168 0258 0668 0290 0655 0349 0424 0053 0546 0045 0039 0005 0059 0652 0347 0706 0216 0349 0732 0764 0271 0955 0283 0875 0476 1259 0732 1408 0372 0955 1567 1784 0432 1562 1182 1940 0269 1865 2017 2745 0107 St Dev 1000 1000 1366 0365 A person with a positive body weight indeX Z2 is smaller and heavier than the average person with the same height Weight indeX Z1 A person with a negative body weight indeX Z2 is taller and lighter than the average person with the same height Weight indeX Z1 Both large positive and large negative values could be a cause for concern EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 151 STATISTICAL REVIEW Geometric Interpretation Scatter plot of Body Weight lndex Zz against Height Weight lndex Zl 3390 We have created two linear 20 Z2 combinations 1 and 2 each of which is interpretable 10 as the vector whose elements o are projections of points onto 00 39 o I a 39 J 0 a directed line segment O 1 0 o The relative position of each Z1 data point has not changed in 20 the new coordinate system with 1 and 2 as its unit base 30 vectors 30 20 10 00 10 20 30 The effect of the matrix multiplication here is a rotation while preserving size and shape EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 152 STATISTICAL REVIEW Geometric Interpretation The values z1 and zz for every point form the new coordinates in the coordinate system with Q1 and wz as its base vectors 0 The rotation matrix 0707 0707 W 0707 0707 gt corresponds to a counterclockwise rotation of the axes by 45 degrees In general we can accomplish a clockwise orthogonal rotation through any angle 0 via the following matrix W cos6 s1n6gt sin0 cos 6 0 An orthogonal rotation matrix has special properties WWTIltgtWT W 1 0 Not all matrix operations have to involve orthogonal rotations EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 153 STATISTICAL REVIEW Geometric Interpretation 0 Matrix operations that are rotations are utilized in principal component analysis and factors analysis To be discussed in a future class 0 Recall that the original variables X15 and X 25 were standardized to have variance 1 This ensures that one variable does not overshadow the other 0 The newly created variables height weight indele and body weight index Zz for which we have the observations g1 and g2 organized in a matrix Z g1 g2 do not have variance 1 anymore 0 Of course we can re standardize the variables Zl and Z2 This can be done by multiplying the matrix Z g1 g2 with a diagonal matrix D1 with the elements 1 8i on the unit diagonal where Si is the standard deviation of the variable Zi Thus 1 18Z 1 0 1 D 0 182gtZs ZD The standardized variables Z 51 and Z 52 will have standard deviation 1 EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 154 STATISTICAL REVIEW Geometric Interpretation Histogram of HeightWeithndex 21 Histogram of BodyWeightIndex 22 18 St Dev 136 16 St Dev 036 14 12 o I so I Frequency Frequency The histogram Z1 describes differences between persons in terms of their height weight index 0 The histogram Z2 describes differences between persons in terms of their body weight index EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 155 STATISTICAL REVIEW Geometric Interpretation The next slide provides a scatter plot of the standardized data set 23201 51 30 20 10 39 30 20 10 00 10 20 30 The shape of the con guration changes from elliptical to circular EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 156 STATISTICAL REVIEW Singular Value Decomposition 0 We started with a data matrix X that contained two arbitrarily scaled and in this case highly correlated variables X 51 and X 52 0 We discovered that we could use one matrix operation to rotate the configuration and another one to change the shape by stretching and shrinking it along different axes 0 We began with an angled ellipse shaped data at a 45 degree angle and ended up with something circular in nature Z3 ZD 1 ltgt Z3 XWD 1 ltgt ZsD XW ltgt X ZsDWT 0 What has been shown although not formally that any data matrix X can be decomposed into three component parts a matrix of uncorrelated variables Z s that have unit variance a stretching and shrinking transformation D and an orthogonal rotation The process of finding these three components is called singular value decomposition SVD and is used in a variety of multivariate data analysis techniques EMSE 171271 FALL 2005 JR van Dorp 101205 dorpjrgwuedu Page 157

×

×

### BOOM! Enjoy Your Free Notes!

×

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

Jim McGreen Ohio University

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Allison Fischer University of Alabama

#### "I signed up to be an Elite Notetaker with 2 of my sorority sisters this semester. We just posted our notes weekly and were each making over \$600 per month. I LOVE StudySoup!"

Jim McGreen Ohio University

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Parker Thompson 500 Startups

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

Become an Elite Notetaker and start selling your notes online!
×

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com