×

### Let's log you in.

or

Don't have a StudySoup account? Create one here!

×

or

by: Isaac Hauck

57

0

14

# Information Theory EEL 3531

Isaac Hauck
University of Central Florida
GPA 3.7

Staff

These notes were just uploaded, and will be ready to view shortly.

Either way, we'll remind you when they're ready :)

Get a free preview of these Notes, just enter your email below.

×
Unlock Preview

COURSE
PROF.
Staff
TYPE
Class Notes
PAGES
14
WORDS
KARMA
25 ?

## Popular in Electrical Engineering

This 14 page Class Notes was uploaded by Isaac Hauck on Thursday October 22, 2015. The Class Notes belongs to EEL 3531 at University of Central Florida taught by Staff in Fall. Since its upload, it has received 57 views. For similar materials see /class/227672/eel-3531-university-of-central-florida in Electrical Engineering at University of Central Florida.

×

## Reviews for Information Theory

×

×

### What is Karma?

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/22/15
FLIGHT 39 Entropy of symboIVBlocgs and Chain Rule 339 A W Overview The entropy of symbol blocks considers the information content of a block of symbols rather than one symbol after another In this consideration the entropy is a joint entropy where its value depends on the joint probability ofthe symbols in the block Overview Ifthe symbols in the sequence or block are statistically independent the source is called a quotmemoryless sourcequot meaning it doesn39t quotrememberquot from one time to the next what symbols it has previously emitted Overview The Chain Rule for Entropy says that the entropy for a symbol block is equal to the entropy ofthe first symbol in the alphabet plus the entropy of the second symbol in the alphabet given the first symbol and so on through the entropy ofthe last symbol given all symbols in the alphabet before that HAo A1 An1 HAo HA1Ao HA2AO A1 I 39All A I Overview Since the entropy of source B given source A is less than or equal to the entropy of source B the entropy of an entire symbol block is less than or equal to the entropy of an entire alphabet from which all the symbols in the symbol block are drawn but they are equal only ifthe source is memoryless W The Symbol Blocks In using the Huffman Code its distribution is inefficient The symbol has a large probability that is not well coded in Huffman coding W The Symbol Blocks This is fixed by replacing the set of symbols m by mAq symbols from packing the symbols by blocks of q Used by m2 for binary alphabet By doing this it breaks the large probability into smaller probabilities that produce the D Shannon entropy bound By encoding source symbols in blocks of engthL you can get to within 1L binary if M J W Entropy Coding Source alphabet prototype vectors a1aM Source probability distribution p1pm Source Entropy H melog2pm bitssymbol Code word lengths l1lm Average codeword length I meIm W Chain Rule I Chain rule HXY HXHYX HYHXY I For random variables X1Xn we have the Chain rule I Think the amount of information that you obtain by observing X1Xn equals the X1 information HXl plus the additional X2 information HX2X1 et cetera I Notice the similarity with the multiplicative rules for joint probabilities pxy pxpyx Reference 125113132searchqcache iFEJ6KlszlJwwwcsucsbe da mteach i ngSo CS 2a5W ee kQI CSZZE pptgrentrgo pyof ml Current Use in Information Technology I Currently this concept is mostly used with Lossless Encryption I Random NumberGeneration I Video Encoding I Mainly the concept helps eliminate bit redundancy and improves data recovery Reference httpbooksgoogecombooksidLEjJYki9UowCamppgPA112amppgPA112ampdqentro pysymbobocksampsourcewebampotsKrSsskgKampsig1de4GiBG jthch3L2NBcl34amphenampsaXampoibookresultampresnum3ampctresultPPPLML htt ok comb 39 L VEBk A1 39 L 39 lp dq trop Lossless Encryption I Helps create a more straightforward encryption algorithm I Allows data recovery by reading blocks of data symbols at a time each block being independent from the next I Creates smallerfiles by utilizing compression I Often used with video encoding and also to improve video and image quality References httpbmrcberkeleyeduresearchmpegfaqmpegzv38faqv38html httpbooksgooglecombooksidLEjJYki9UowC8 pgPA1128 lpgPA112ampdqentropy symbo lblocksampsourcewebampotsKrSss k Kampsi 1D dp4GiBG Video Encoding I The Entropy concepts of symbol blocks and the Chain Rule are mostly used with video compression I Video quality enhancement by reducing distortion over the transmission channel This also is applicable to image processing and transmission over wireless channels References httpwwwedncoma rticleCA339712html httpbooksgooglecombooksidOYFYt5C4N94CamppgPA648amplpgPA648ampdqentropy symbobockschainrUleampsourcewebampotsuquVi Mc9 Mampsigk82Md RRRbABYj xE yRaZJ r2042 Iamp hjle39 nampls a oi boo kre su lt81r e39sn u mlrzt8cctresult Future Development I HiDefinition Video Streaming over H264 Channel Applications Image distortion removal I The concept is also been applied in IT management as a way to eliminate redundant tasks inside ofthe IT FESOUFCES References httpwwwieeecacanrevcanrev4ocotepdf httpbmrcberkeleyedUresearchmpegfaqmpeg2v38faqv38htm bouma nece438Iecturemodue225i magyecom pres Vdf Other References I httpwwwsciencedirectcomscienceobArticeURLampUdiBSH14 4TPGNHV HampUser1oamprdoc1ampfmtamporigsearchampsortdampviewcampacctCoooo50221ampversion1amp urIVe rsionoampuserid1oampmd55obf525715c20660591659c97c3b9c4a I httpnowpublisherscomproductaspxdoi0100000013ampproductClT I httpwwwciphersbvrittercomG LOSSARYHTM I httgzuenwiki9ediaorgwikinformation entrogy I httpiphomehhidewieqandassetspdfsDIC information entropv o7pdf I httgwwwmdpicoms ea39rchlqENTROPY8 S iOUrnaoamps volume18zxs aUthors8 s sectiono 8 s i sf s Ueamps article ti je3l8gss e c ifal isfsvueoampisquot ta quot39e

×

×

### BOOM! Enjoy Your Free Notes!

×

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

Jim McGreen Ohio University

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Allison Fischer University of Alabama

#### "I signed up to be an Elite Notetaker with 2 of my sorority sisters this semester. We just posted our notes weekly and were each making over \$600 per month. I LOVE StudySoup!"

Jim McGreen Ohio University

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Parker Thompson 500 Startups

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

Become an Elite Notetaker and start selling your notes online!
×

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com