### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# 658 Class Note for STAT 59800 with Professor Neville at Purdue

### View Full Document

## 18

## 0

## Popular in Course

## Popular in Department

This 14 page Class Notes was uploaded by an elite notetaker on Friday February 6, 2015. The Class Notes belongs to a course at Purdue University taught by a professor in Fall. Since its upload, it has received 18 views.

## Similar to Course at Purdue

## Reviews for 658 Class Note for STAT 59800 with Professor Neville at Purdue

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 02/06/15

Data Mining 3857300 STAT 59800 024 Purdue University February 19 2009 Support vector machines Support vector machines 0 Discriminative model 9 General idea 9 Find best boundary points support vectors and build classifier on top of them 0 Linear and nonlinear SVivls Choosing hyperplanes Maximize margin In Bl bll margin lor 81 D12 Source lnlroduclionlo DalaMining Tan Sieinbacn and Kurmr Linear SVMs a Search for hyperplane with largest margin a Margind 317 support vectors where d is distance to closest positive example and Cl is distance to closest negative example Constrained optimization 52 1f0ryj 1 b 1 for yj 71 l Eq3 yjj w 712 OVyU qu Eq21 96039 71 71 H1 3jwb1 H2 3jwb71 1 i d d margin iiwii 7 iiwii 3 Can maximize margin by minimizing w2 subject to constraints Constrained optimization 0 Can maximize margin by minimizing w subject to constraints on EqS Introduce Lagrange multipliers and minimize Lagrange function I I 1 Lp w2 gemaim w b 21 21 Minimize LP with respect to w b and on 20 Convex programming problem I LB Em e gZazajymwxm wmi i1 23339 Dual problem maximize LD wrt constraints on w b and on 20 Dual problem 0 For a convex problem no local minima the dual problem is equivalent to the primal problem ie we can switch between them O Dual depends on inner product between feature vectors simpler quadratic programming problem 10 Limitations of linear SVMs O Linear classifiers cannot deal with Non linear concepts O Noisy data 0 Solutions 0 Soft margin eg allow mistakes in training data C Network of simple linear classifiers eg neural networks 0 Map data into richer feature space egq non linear features and then use linear classifier 11 Map to new features O Define a new set of features where data are linearly separable ltlgtXX1 X2 X1X2 X12 X22 12 Kernel trick Note that the dual problem only depends on iT i 0 Move to an infinite number of features by replacing I with a kernel iT i 39 KLi 0 Here kernel K is a function that returns the value ofthe dot product between the two arguments 0 As long as kernel is symmetric and positive semi definite you can forget about the features 0 Example Polynomial kernel KiJr XiTxjd 13 Kernel SVMs 0 State of the art classifier with good kernel O Solves computational problem of working in high dimensional space 0 Non parametric classifier keeps all data around in kernel O Learning On approximations available for On 14 Predictive modeling evaluation 15 Empirical evaluation 0 Given observed accuracy of a model on limited data how well does this estimate generalize for additional examples O Given that one model outperforms another on some sample of data how likely is it that this model is more accurate in general 0 When data are limited what is the best way to use the data to both learn and evaluate a model 16 Score functions 39 Zeroone loss N as Accuracy sltMgt i dime M gm 0 Sensitivityspecificity i1 PrecisionRecallFl Actual 39 Absolute loss 39 Squared loss E TP FP Root meansquared error FN TN 0 I Likelihoodconditional likelihood I Area under the ROC curve 17 Algorithm comparison 0 How to compare the performance of two learning algorithms A and B 0 Estimate the expected value of the difference in errors where expectation is over all datasets D of size N 39 In practice we only have a limited sample Do 18 Approach 0 Use kfold crossvalidation to get k estimates of error for MA and ME Mean is estimate of expected error 0 Use paired ttest to assess whether the two distributions are statistically different from each other Issues I Assumes errors for all individuals are equally weighted May want to weight recent instances more heavily 0 May want to include information about reliability of sets of measurements Assumes errors for all contexts are equally weighted I May want to weigh false positive and false negatives differently May be costs associated with acting on particular classifications eg marketing to individuals Examples 0 Loan decisions 0 Cost of lending to a defaulter is much greater than the lost business of refusing loan to a non defaulter 0 Oil slick detection O Cost of failing to detect an environmental disaster is far less than the cost of a false alarm 0 Promotional mailing 0 Cost of sending junk mail to a household that doesn t respond is far less that the lost business of not sending it to a household that would have responded 21 Costsensitive models 0 Define a score function based on a cost matrix 0 If y is the predicted class and y is the true class then need to define a Aetual matrix of costs Cyy g TP FP 0 Reflects the severity of classifying an instance with true class y to class y E FN TN 0 22 ROC curves O Receiver Operating Characteristic curve q 0 Plots the true positive rate 3 against the false positive rate for different classification thresholds 9 37 E D F s Evaluates performance over varying 3 costs and class distributions N 0 Base RPT O Can summarize With area under 3 l l l l l l theCUrVeAUC 00 02 04 06 08 10 FP Rate Biasvariance analysis EDL5qty EDI EDI2 EDI EDy2 EDEDy y2 noise bias variance 39 Noise loss incurred independent of algorithm Bias loss incurred of mean prediction relative to optimal Variance average loss of predictions compared to mean prediction Biasvariance analysis A Model predictions Test Set Training 5 e1 Samples Models 25 Findings 0 Bias 0 Often related to size of model space 0 More complex models tend to have lower bias 0 Variance Often related to size of dataset 0 When data is large enough to estimate parameters well then models have lower variance 0 Simple models can perform surprisingly well due to lower variance 26 Bxasvamance tradeo m mp mmmm Expected NEE Sis m paramslsv space 27 Next c ass Tupm Eaggmg amd mman Pathumgwes m earmng a gunthms 2a

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "I made $350 in just two days after posting my first study guide."

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.