×

### Let's log you in.

or

Don't have a StudySoup account? Create one here!

×

or

## INFORMATION THEORY

by: Deondre Ullrich

27

0

1

# INFORMATION THEORY ELEC 535

Deondre Ullrich
Rice University
GPA 3.54

Staff

These notes were just uploaded, and will be ready to view shortly.

Either way, we'll remind you when they're ready :)

Get a free preview of these Notes, just enter your email below.

×
Unlock Preview

COURSE
PROF.
Staff
TYPE
Class Notes
PAGES
1
WORDS
KARMA
25 ?

## Popular in Electrical Engineering & Computer Science

This 1 page Class Notes was uploaded by Deondre Ullrich on Monday October 19, 2015. The Class Notes belongs to ELEC 535 at Rice University taught by Staff in Fall. Since its upload, it has received 27 views. For similar materials see /class/224979/elec-535-rice-university in Electrical Engineering & Computer Science at Rice University.

×

## Reviews for INFORMATION THEORY

×

×

### What is Karma?

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/19/15
14 The Data Processing Theorem The Data Processing Theorem compares the Kullback Leibler distance relative entropy be tween the probability distributions of two inputs to a system to the distance between the cor responding output distributions The Data Processing Theorem says the ratio of the output distance to the input distance must be less than or equal to one Suppose Xn is a sequence of statistically independent Gaussian random variables having some mean and variance 0 The system simply adds Nn white Gaussian noise statistically independent of the input having zero mean and variance 0 to produce the output Yn Xn N Here we explore the Data Processing Theorem when the input distributions dilTer only in their means a If Xn has mean me or m1 what is the ratio of the output to the input KullbackLeibler distances b The Data Processing Theorem does not suggest whether feedback can increase or decrease the ratio of distances Suppose YnaYn71XnNn7 indicating the previous output modi es the value of each current output For Yn to be stationary the coef cient must satisfy lal lt 1 What is the ratio of KullbackLeibler distances between the output and input values at some time index no Can the ratio be greater than one c V More generally we can study Y AXN which represents amore general expression for feedback Assume that A is an invertible matrix and that X and N are Gaussian random vectors statistically independent of each other with covariance matrices 031 and UJZVI respectively As before N has zero mean and the mean of X assumes two possible values What is the ratio of KullbackLeibler distances between the output and input vectors Comment on your answer in light of your result for part b

×

×

### BOOM! Enjoy Your Free Notes!

×

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

Bentley McCaw University of Florida

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

Janice Dongeun University of Washington

#### "I used the money I made selling my notes & study guides to pay for spring break in Olympia, Washington...which was Sweet!"

Jim McGreen Ohio University

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Parker Thompson 500 Startups

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

Become an Elite Notetaker and start selling your notes online!
×

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com