### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# 576 Class Note for STAT 597I at PSU

### View Full Document

## 22

## 0

## Popular in Course

## Popular in Department

This 14 page Class Notes was uploaded by an elite notetaker on Friday February 6, 2015. The Class Notes belongs to a course at Pennsylvania State University taught by a professor in Fall. Since its upload, it has received 22 views.

## Similar to Course at Penn State

## Popular in Subject

## Reviews for 576 Class Note for STAT 597I at PSU

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 02/06/15

Mixture Discriminat Analysis J15 L1 http www Sta Mixture Discriminat Analysis Jia Li Department of Statistics The Pennsylvania State University Emaii panama psu edu http www stat psu eduth Mixture Di minal Analysis Mixture Discriminant Analysis gt A method for classification supervised based on mixture models gt Extension of linear discriminant analysis gt The mixture of normals is used to obtain a density estimation for each class J15 L1 httpWWWstatpsueduleall Mixture Discriminal Analys Linear Discriminant Analysis gt Suppose we have K classes gt Let the training samples be X17 Xn with classes 217 zn z E 17 K gt Each class with prior probability ak is assumed to follow a Gaussian distribution gtXl k7X gt Model estimation 271 Zi k n 271 Xi Zi k k 2 Iz k 3k 2 21100 dzXXI IU Zt J15 L1 httpWWWstatpsueduleall Mixture Discriminal Analysis gt Given a test sample X X the Bayes classification rule is 2 arg mfx 3k xi k7 X The decision boundary is linear because X is shared by all the classes J15 L1 httpwwwstatp Mixture Di minal Analysis Mixture Discriminant Analysis gt A single Gaussian to model a class as in LDA is too restricted gt Extend to a mixture of Gaussians For class k the within class density is Rx fkX Z Wkr xi kr7 X r1 gt A common covariance matrix is still assumed J15 L1 httpWWWstatpsueduleall A 2classes example Class 1 is a mixture of 3 normals and class 2 a mixture of 2 normals The variances for all the normals are 30 000 000 007 005 005 00A 003 002 001 ha L httpwwwstatpsueduleah Mixture Discriminal Analys Model Estimation gt The overall model is Rk PX x z k amx ak Zwmmpkn r1 where ak is the prior probability of class k gt The ML estimation of ak is the proportion of training samples in class k gt EM algorithm is used to estimate mm Ink and X gt Roughly speaking we estimate a mixture of normals by EM for each individual class gt X needs to be estimated by combining all the classes J15 L1 httpWWWstatpsueduleall Mixture Di minal Analysis gt EM iteration gt Estep for each class k collect samples in this class and compute the posterior probabilities of all the Rk components Suppose sample i is in class k 39 2 Ply Rrkr xllllkn r 1MRlt E1 7Tk xiler7 gt Mstep compute the weighted MLEs for all the parameters 21 Zi kPir 271 2139 k Dquot Xilzi kPir My n Ei1lzi kPir R1 2 21 2amp1 PirXi 2 MN n J15 L1 httpWWWstatpsueduleall Waveform Exa m ple gt Three funcuons mm mm mm are smned Versmns ofezch omen as shown m the gure Each h sspecmed by the eqquMerz Mghttmng efuncuon Ms vames 2L mtegers 1 1N 212re measured h l m h 3 0 m u m m m w sinij2 Mixture Di minal Analysis gt The three classes of waveforms are random convex combinations of two of these waveforms plus independent Gaussian noise Each sample is a 21 dimensional vector containing the values of the random waveforms measured at 739 1221 gt To generate a sample in class 1 a random number u uniformly distributed in 01 and 21 random numbers 61 62 621 normally distributed with mean zero and variance 1 are generated Xj uh1j17 uh2j a j 1111211 gt To generate a sample in class 2 repeat the above process to generate a random number u and 21 random numbers 61 621 and set Xj uh1j17 uh3j a j 1111211 gt Class 3 vectors are generated by Xj uh2j17 uh3j 6137 j 1111211 J15 L1 httpWWWstatpsueduleall Mixture Discriminal Analysis Example random waveforms E 2 39 f 4 39 39 N 4 g 39 2 39 G D u 39 39 72 72 1 74 u 5 1m 15 2n n 5 1m 15 2n 6 a I B 4 I E 4 5 39 2 52 u D 39 72 72 74 u 5 1m 15 2D 5 1m 15 in B 4 39 39 39 5 5 u u 39 2 39 4 5 5 1m 15 2n n 5 15 2D J15 L1 htt Mixture Discriminat Analysis mm m Suusnul Lennmlx uum TAhllurnm 2 F nedmnn znm Chum 12 mass 1 Class 3 Figure 1212 Some examples of the wavefums gan emted from mudel 1262 before the Gaussian noise is added Jia Li htt 39www stat psu eduNjiali Mixture Discriminal Analysis gt A three component mixture of normals is assumed for each class gt The Bayes risk has been estimated to be about 014 gt MDA outperforms LDA and QDA gt Low dimension views are obtained from projecting on to canonical coordinates J15 L1 httpwwwstatp Mixture Discriminat Analysis E emmlu m Scnuanc Leann anne Tlhllurm a Friedmnn 20m Ilamu 12 asmasseswananmm Ssubdas 5Fenahzedddr a a w gt gt a E E a N E a s 4 2 n 2 4 s 2 u 2 nacnmnam Vam unanmmm Var a Figure 1213 Same mada menaa andz niewa of the MDA model tted w a sumpla of the waveform model The points 1175 independent test data pmjected an to the leading and canonical cum39dinntes za yanel and the thde and femm quotyin panaz The subclass cantan m indicated Jia Li http39www stat psu eduNjiali

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "I made $350 in just two days after posting my first study guide."

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.