New User Special Price Expires in

Let's log you in.

Sign in with Facebook


Don't have a StudySoup account? Create one here!


Create a StudySoup account

Be part of our community, it's free to join!

Sign up with Facebook


Create your account
By creating an account you agree to StudySoup's terms and conditions and privacy policy

Already have a StudySoup account? Login here

Theoretical Statistics

by: Floy Kub

Theoretical Statistics STAT 210A

Floy Kub

GPA 3.64


Almost Ready


These notes were just uploaded, and will be ready to view shortly.

Purchase these notes here, or revisit this page.

Either way, we'll remind you when they're ready :)

Preview These Notes for FREE

Get a free preview of these Notes, just enter your email below.

Unlock Preview
Unlock Preview

Preview these materials now for free

Why put in your email? Get access to more of this material and other relevant free materials for your school

View Preview

About this Document

Class Notes
25 ?




Popular in Course

Popular in Statistics

This 3 page Class Notes was uploaded by Floy Kub on Thursday October 22, 2015. The Class Notes belongs to STAT 210A at University of California - Berkeley taught by Staff in Fall. Since its upload, it has received 38 views. For similar materials see /class/226735/stat-210a-university-of-california-berkeley in Statistics at University of California - Berkeley.


Reviews for Theoretical Statistics


Report this Material


What is Karma?


Karma is the currency of StudySoup.

You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/22/15
Stat210B Theoretical Statistics Lecture Date February 8 2007 Lecture 7 Lecturer Michael I Jordan Scribe Kurt Miller 1 Properties of VCClasses 11 VC preservation Let C and D be VCclasses Le classes With nite VCdimension Then so are CE C e C o CUD1CECDED o C D1CECDED o Where lt1 is ll o CXD2CECDED 12 Half spaces Let Q be a nitedimensional vector space of functions Let C g 2 0 g E Q or more formally C w 9a 2 0 g E Q Then V0 S dimQ l 13 Subgraphs De nition 1 A subgmph of f X A R is the subset X X R given by 95 t t S A collection 7 is a VCsubgmph class if the collection of subgraphs is a VC class 2 Covering Number We now begin to explore a more powerful method of de ning complexity than VCdimension 21 De nitions De nition 2 Covering Number Pollard 1984 p 25 Let Q be a probability measure on S and f be a class of functions in L1Q ie Vf E f f lt 0 For each E gt 0 de ne the L1 covering number N1E Q 7 as the smallest value of m for Which there exist functions 91 gm not necessarily in f such that minj E 9quot S E for each f in 7 For de niteness set N1E Q 7 0 if no such m exists 2 Lecture 7 Note that the set 9 that achieves this minimum is not necessarily unique De nition 3 Metric Entropy De ne H1E Qf log N1 E Q 7 as the 1 metric entropy of 7 More generally HpE Qf uses the LpQ norm Write this as lg lm flglde1plt De nition 4 Totally bounded A class is called totally bounded if VE HpE Qf lt 00 Another kind of entropy De nition 5 Entropy with bracketing Let N BE Q 7 be the smallest value of m for which there exist pairs of functions gf such that Vj 179lepr lt E and Vf E f 3jf st 9 S f 3 gym Then we de ne the entropy with bracketing as prQE Qf log prQE Qf Finally using IgHoo supxgx gx let NOOE7 be the smallest m such that there exists a set g l such that supfef minj1wym 7 ngoo lt E Then HOOET lOgNoQE f 22 Relationship of the various entropies Using the de nitions above we have that 1 H1EQf s H BEQf Va gt 0 2 HPBEQ S HooE27 VE gt 0 Can these quantities be computed for normal classes of functions Yes but you would generally look them up in a big book We ll look at how to compute one of these quantities here 23 Examples Example 6 Let f f 01 l S l ie functions from 01 to 01 with rst derivatives bounded by 1 Then H00 E f where A is a constant that we will compute ll0 A Proof Let 0 10 lt 11 lt lt am l where we kE and k 0m Let Bl 10111 and Bk ak71ak For each f E 7 de ne N m flllc f 5 lBC k21 f takes on values in Ek where k is an integer We also have 3 2E because fak1 7 fak1 S E by construction and fac 7 fak1 S E since f is bounded by 1 We now count the number of possible f obtained by this construction At 10 there are lE 1 choices for fa0 since f only takes on values of Ek in 0 1 Furthermore combining previous results gives us Ware 7 aker Wale ak mate fak71llfak71i aker S 3 3E Lecture 7 3 Therefore having chosen fak1 f can take on at most 7 distinct values at ak Therefore Nooaaf s 1 71 which gives us that 1 Hoo25f S E log7 logl1Ej 1 so our constant can be chosen as any constant that gt log 7 El A seminal paper in this eld is by Birman and Solomjak in 1967 They present other examples of metric entropy calculations including Example 7 Let f f 01 a 01 ffltmgtx2dx g 1 Then Hagar g flailm Example 8 Let f f R A 01 f isincreasing Then H BEQ S A Example 9 Let R A 01 f S 1 the class of bounded variation Then HpBE Qf 3 Ag Lemma 10 Ball covering lemma A ball BAR in Rd of radius R can be covered by 4R E d E Proof Let Cj be a packing of size E Euclidean norm This implies that balls of radius E with centers at Cj cover BAR otherwise we could add more points Cj to the packing Let Bj be the ball of radius 54 centered at cj We must have that B O Bj is empty for i y j Therefore 3 are disjoint and balls of radius E Ujsj c BdRE4i A ball of radius p has volume Cdpd where Cd is a constant that depends on the dimension d Therefore the volume of the union Uij is MCdE4d and since it is a subset of BAR 54 we have E d E d lt i Mcd lt4 Cd RT 4 With a simple manipulation of this equation we get that a MS lt4REgt E References Pollard D 1984 Convergence of Stochastic Processes Springer New York


Buy Material

Are you sure you want to buy this material for

25 Karma

Buy Material

BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.


You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

Why people love StudySoup

Steve Martinelli UC Los Angeles

"There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

Allison Fischer University of Alabama

"I signed up to be an Elite Notetaker with 2 of my sorority sisters this semester. We just posted our notes weekly and were each making over $600 per month. I LOVE StudySoup!"

Bentley McCaw University of Florida

"I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"


"Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!

Refund Policy


All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email


StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here:

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.