### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Theoretical Statistics STAT 210A

GPA 3.64

### View Full Document

## 38

## 0

## Popular in Course

## Popular in Statistics

This 3 page Class Notes was uploaded by Floy Kub on Thursday October 22, 2015. The Class Notes belongs to STAT 210A at University of California - Berkeley taught by Staff in Fall. Since its upload, it has received 38 views. For similar materials see /class/226735/stat-210a-university-of-california-berkeley in Statistics at University of California - Berkeley.

## Reviews for Theoretical Statistics

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/22/15

Stat210B Theoretical Statistics Lecture Date February 8 2007 Lecture 7 Lecturer Michael I Jordan Scribe Kurt Miller 1 Properties of VCClasses 11 VC preservation Let C and D be VCclasses Le classes With nite VCdimension Then so are CE C e C o CUD1CECDED o C D1CECDED o Where lt1 is ll o CXD2CECDED 12 Half spaces Let Q be a nitedimensional vector space of functions Let C g 2 0 g E Q or more formally C w 9a 2 0 g E Q Then V0 S dimQ l 13 Subgraphs De nition 1 A subgmph of f X A R is the subset X X R given by 95 t t S A collection 7 is a VCsubgmph class if the collection of subgraphs is a VC class 2 Covering Number We now begin to explore a more powerful method of de ning complexity than VCdimension 21 De nitions De nition 2 Covering Number Pollard 1984 p 25 Let Q be a probability measure on S and f be a class of functions in L1Q ie Vf E f f lt 0 For each E gt 0 de ne the L1 covering number N1E Q 7 as the smallest value of m for Which there exist functions 91 gm not necessarily in f such that minj E 9quot S E for each f in 7 For de niteness set N1E Q 7 0 if no such m exists 2 Lecture 7 Note that the set 9 that achieves this minimum is not necessarily unique De nition 3 Metric Entropy De ne H1E Qf log N1 E Q 7 as the 1 metric entropy of 7 More generally HpE Qf uses the LpQ norm Write this as lg lm flglde1plt De nition 4 Totally bounded A class is called totally bounded if VE HpE Qf lt 00 Another kind of entropy De nition 5 Entropy with bracketing Let N BE Q 7 be the smallest value of m for which there exist pairs of functions gf such that Vj 179lepr lt E and Vf E f 3jf st 9 S f 3 gym Then we de ne the entropy with bracketing as prQE Qf log prQE Qf Finally using IgHoo supxgx gx let NOOE7 be the smallest m such that there exists a set g l such that supfef minj1wym 7 ngoo lt E Then HOOET lOgNoQE f 22 Relationship of the various entropies Using the de nitions above we have that 1 H1EQf s H BEQf Va gt 0 2 HPBEQ S HooE27 VE gt 0 Can these quantities be computed for normal classes of functions Yes but you would generally look them up in a big book We ll look at how to compute one of these quantities here 23 Examples Example 6 Let f f 01 l S l ie functions from 01 to 01 with rst derivatives bounded by 1 Then H00 E f where A is a constant that we will compute ll0 A Proof Let 0 10 lt 11 lt lt am l where we kE and k 0m Let Bl 10111 and Bk ak71ak For each f E 7 de ne N m flllc f 5 lBC k21 f takes on values in Ek where k is an integer We also have 3 2E because fak1 7 fak1 S E by construction and fac 7 fak1 S E since f is bounded by 1 We now count the number of possible f obtained by this construction At 10 there are lE 1 choices for fa0 since f only takes on values of Ek in 0 1 Furthermore combining previous results gives us Ware 7 aker Wale ak mate fak71llfak71i aker S 3 3E Lecture 7 3 Therefore having chosen fak1 f can take on at most 7 distinct values at ak Therefore Nooaaf s 1 71 which gives us that 1 Hoo25f S E log7 logl1Ej 1 so our constant can be chosen as any constant that gt log 7 El A seminal paper in this eld is by Birman and Solomjak in 1967 They present other examples of metric entropy calculations including Example 7 Let f f 01 a 01 ffltmgtx2dx g 1 Then Hagar g flailm Example 8 Let f f R A 01 f isincreasing Then H BEQ S A Example 9 Let R A 01 f S 1 the class of bounded variation Then HpBE Qf 3 Ag Lemma 10 Ball covering lemma A ball BAR in Rd of radius R can be covered by 4R E d E Proof Let Cj be a packing of size E Euclidean norm This implies that balls of radius E with centers at Cj cover BAR otherwise we could add more points Cj to the packing Let Bj be the ball of radius 54 centered at cj We must have that B O Bj is empty for i y j Therefore 3 are disjoint and balls of radius E Ujsj c BdRE4i A ball of radius p has volume Cdpd where Cd is a constant that depends on the dimension d Therefore the volume of the union Uij is MCdE4d and since it is a subset of BAR 54 we have E d E d lt i Mcd lt4 Cd RT 4 With a simple manipulation of this equation we get that a MS lt4REgt E References Pollard D 1984 Convergence of Stochastic Processes Springer New York

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "I signed up to be an Elite Notetaker with 2 of my sorority sisters this semester. We just posted our notes weekly and were each making over $600 per month. I LOVE StudySoup!"

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.