### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Reinforcement Learning in Artificial Intelligence ECE 517

UT

GPA 3.6

### View Full Document

## 13

## 0

## Popular in Course

## Popular in Electronics and Computer Technology

This 13 page Class Notes was uploaded by Jessica Nienow II on Monday October 26, 2015. The Class Notes belongs to ECE 517 at University of Tennessee - Knoxville taught by Staff in Fall. Since its upload, it has received 13 views. For similar materials see /class/229858/ece-517-university-of-tennessee-knoxville in Electronics and Computer Technology at University of Tennessee - Knoxville.

## Reviews for Reinforcement Learning in Artificial Intelligence

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/26/15

MATLAB Tutorial You need a small number of basic commands to start using MATLAB This short tutorial describes those fundamental commands You need to create vectors and matrices to change them and to operate with them Those are all short high level commands because MATLAB constantly works with matrices I believe that you will like the power that this software gives to do linear algebra by a series of short instructions create E create u change E multiply Eu E eye3 u E 1 E31 5 v E gtlt u 1 0 0 1 1 0 0 1 0 1 0 0 0 1 0 0 0 0 1 0 5 0 1 5 The word eye stands for the identity matrix The submatrix u E z 1 picks out column 1 The instruction E 3 1 5 resets the 3 1 entry to The command E gt1lt u multiplies the matrices E and u All these commands are repeated in our list below Here is an example of inverting a matrix and solving a linear system create create 3 invert solve l ones3 eye3 l A 3 C invA x AV or x C gtlt l 2 1 1 1 75 25 25 0 1 2 1 1 25 75 25 0 1 1 2 2 25 25 75 1 The matrix of all ones was added to eye3 and l is its third column Then invA produces the inverse matrix normally in decimals for fractions use format rat The system l is solved by x invA gtlt l which is the slow way The backslash command x AV uses Gaussian elimination if is square and never computes the inverse matrix When the right side I equals the third column of A the solution 3 must be 0 0 1 The transpose symbol I makes 42 a column vector Then picks out the third column of A and we have I Here are a few comments The comment symbol is The symbols a and are allierent MATLAB is case sensitive Type help slash for a description of how to use the backslash symbol The word help can be followed by a MATLAB symbol or command name or M file name Note The command name is upper case in the description given by help but must be lower case in actual use And the backslash AV is different when is not square To display all 1639 digits type quot0771in long The normal quot0771in short gives 4 digits after the decimal A sei nicolon after a command avoids display of the result ones3 will not display the 3 X 3 identity matrix SQ the Up dl l OVV cursor to return 0 previous commands How to input a row or column vector u 2 4 has one row with three coi nponents 1 X 3 matrix v 22 4 has three rows separated by sei39nicolons 3 x 1 matrix v 2 4 5 or v u transposes u to produce the same v w 25 generates the row vector w 2 3 4 with unit steps u 1217 takes steps of 2 to give u 1 3 5 7 How to input a matrix a row at a time 1 2 3 4 5 has two rows always a sei nicolon between rows 1 2 3 also produces the matrix but is harder to type B 1 2 3 4 5 6 is the transpose of Thus is A in MATLAB How to create special matrices diagv produces the diagonal matrix with vector v on its diagonal toeplitzv gives the syi ni netric constant diayonal matrix with v first row and first col umn toeplitzwv gives the constant diagonal matrix with w first column and v first row onesn gives an n x n matrix of ones gxgmzm mn umgm p s mm 8018 1191 Gas smxa vqu 3g 1g o3 uopmos am saxng AV x uogswp 333x sg 15 015219390q IS3S XG vqu 3g g gtlt vqu saxng 30 aswwgopmsd am saxng vymgd axqpmug pm amtbe sg 3g 14V saxng vying gazgs vngs 3g 33npmd 5x3tm Kq Kmm am saxng g gtlt g Kidmmn um 3g gv 33npmd xmmn am saxng g gtlt Impalamg pue uogqeoudgtunm xgnew mm tmm mmo max 03 mxo mp ugmn am GAOQR samua up 8338 Why mm tmm mddn max 03 mxo mp ugmn am mopq samua up was Wkly xmmn x um 30 sumtqm 319 may pmmo Inmwa 3qu Gun summx xmmn u x z sumtqm up pm 17 pm a 8mm summx p g xmmn g x g m 3 mm sumtqm pm p 03 a mag 8mm summx g p 103mm ummm 30 ummm WI am summx 1 103mm mm 30 mm m am summx xmmn gtlt mpzas xmmn am 30 5mm 11quot am summx 11quotV V XEJQEIII u x w HE J0 SBOEJJEIIIql lS 619613 01 AAOH 30 g ptm a 8mm sa tmqaxa k g 8mm up w summm up 0 m spums aqusz 1mm aql m 1mm 03 umth pumas mp 833831 m a 2V a 1mm 03 mm pulp mp 838861 a 2 1 1mm 03 Sum g g mp 838861 A a V xgnem 11ng 13 ug septum 3311qu 01 MOH aduqs 11st am 30 magnum aAg vazgsa a vazgssoxaz Vazgssauo magnum H X m aAg u mpuex u msoxaz u msauo mumpr pm 0 Imam samua pmnqmsgp Suzzuuou mm xmmn u x u up saAg Mupum uognqmsgp UHO Im pm 0 Imammq samua mopmax mm xmmn u x u up saAg upum xmmn mnuapg H X H am saxng 70912 smaz 30 xmmn H X H UP mng Msoxaz Numbers and matrices associated with A detA is the determinant if is a square matrix rankA is the rank number of pivots dimension of row space and of column space sizeA is the pair of numbers m n traceA is the trace sum of diagonal entries sum of eigenvalues nu11A is a matrix Whose n 7 columns are an orthogonal basis for the nullspace of orthA is a matrix Whose 7 columns are an orthogonal basis for the column space of Examples E eye4 E 2 1 3 creates a 4 X 4 elementary elimination matrix E subtracts 3 times row 1 of from row 2 B 3 creates the augmented matrix with 2 extra column E eye3 P E 1 3 1 creates a permutation matrix Note that triuA trilA diagdiagA equals Builtin M les for matrix factorizations all important L U P 11114 gives three matrices with FA LU a eigA is a vector containing the eigenvalues of 5 eigA gives a diagonal eigenvalue matrix E and eigenvector matrix S With AS SE If is not diagonalizable too few eigenvectors then S is not invertible 9 R qrA gives an m X m orthogonal matrix Q and m X n triangular R With QR Creating M les M files are text files ending With m Which MATLAB uses for functions and scripts A script is a sequence of commands Which may be executed often and can be placed in an m file so the commands do not have to be retvped MATLAB s demos are examples of these scripts An example is the demo called house Most of MATLAB s functions are actually m files and can be viewed by writing type 272727 Where 272727 is the name of the function To write your own scripts or functions you have to create a new text file with any name you like provided it ends with m so MATLAB will recognize it Text files can be created edited and saved with any text editor like cniocs EZ or iii A script file is simply a list of MATLAB commands When the file name is typed at the MATLAB prompt the contents of the file will be executed For an m file to be a function it must start with the word function followed by the output variables in brackets the function name and the input variables Examples function niult A itmnkpl C A gtlt Save the above commands into a text file named multm Then this funtion will take a matrix and return only the matrix product C The variable 39739 is not returned because it was not included an output variable The commands are followed by so that they will not be printed to the MATLAB window every time they are executed It is useful when dealing with large matrices Here is another example function V D 39739pi39opci39tics A 95 This function nds tlic mnk cigcnuolucs and cigcnucctoi s of A ninsizcA if nin VDcigA itmnkpl clsc disp Ei i oi 39 Tlic matrix must bc squm c cnd Here the function takes the matrix A input and only returns two matrices and the rank output The 95 is used a cormnent The function checks to see if the input ma trix is square and then finds the rank eigenvalues and eigenvectors of a matrix A Typing pi opc39rticsm only returns the first output V the matrix of eigenvectors You must type VDipiopciticsA to get all three outputs Keeping a diary of your work The command diary le tells MATLAB to record everything done in the MATLAB window and save the results in the text file named file Typing diary on or diary off toggles the recording Old diary files can be viewed using a text editor or printed using pr in unix In MATLAB they can be viewed using the type le command Saving your variables and matrices The command diary saves the commands you typed well MATLAB s output but it does not save the content of your variables and matrices These variables can be listed by the command Whos which also lists the sizes of the matrices The command save 5008 will save the matrices and all variables listed by the Whos command into the file named 22717 MATLAB labels these files with a mat extension instead of m which are scripts or functions malmat files can be read by MATLAB at a later time by typing load 27272 Graphics The simplest command is plotx g which uses two vectors x and y of the same length The points xi 3 will be plotted and connected by solid lines If no vector x is given MATLAB assumes that xi 27 Then ploty has equal spacing on the x axis the points are kg27 The type and color of the line between points can be changed by a third argument The default with no argument is a solid black line quot lse help plot for many options we indi cate only a few MATLAB 5 plotx y 7 1 plots in 7 red with for points and dotted line MATLAB 4 plotx y is a dashed line and plotx y is a dotted line You can omit the lines and plot only the discrete points in different ways plotxy o gives circles Other options are or x or gtlt For two graphs on the same axes use plotxyXY Replace plot by loglog or semilogy or semilogx to change one or both axes to logarithmic scale The command AMATH 352 Lecture 3 MATLAB Tutorial MATLAB short for MATrix LABoratory is a very useful piece of software for numerical analysis It provides an environment for computation and the visualization Learning MATLAB is not the goal of this course but a working knowledge of MATLAB will allow you to implement and test the algorithms that form the heart of this course Seeing these algorithms at work will hopefully enable a deeper understanding of the mechanics strengths and pitfalls of each algorithm Starting MATLAB To start MATLAB click on the MATLAB icon or type matlab at the command line prompt To exit type quit Entering Variables There are three types of variables that you can use in matlab scalars vectors and matrices To assign a value to a scalar variable type gtgt X 02 After hitting return MATLAB will echo the value of the variable back to you x 02000 If you don t want to see the value of the variable add a semicolon at the end of the line gtgt X 02 To enter a vector or matrix use square brackets to indicate the start and end of the vectormatrix For example a row vector may be entered gtgt y 0 l 2 3 4 y To enter a variable with more than one row the rows of the vector or matrix are separated by semicolons or by carriage returns For example a column vector may be entered gtgt Z 0 l 2 3 4 wwl o A matrix may be entered similarly with the rows of the matrix separated in this case by carriage returns and the whole expression enclosed in square brackets gtgt A 0 2 3 7 12 N A 0 2 3 7 12 8 The complex conjugate transpose of a vector or matrix may be obtained by placing an apostrophe after the expression gtgt W T w 0 1 2 3 4 gtgt B A B 0 3 12 2 7 8 To give a variable a set of evenlyspaced values use the colon operator gtgt t 06 gtgt u 00318 0 03000 06000 09000 12000 15000 18000 The rst and last numbers are the starting and ending points for the series The middle number is the spacing between the members of the series If no spacing is given MATLAB assumes a spacing of l Accessing elements of a vectormatrix The individual elements of a vector or matrix may be accessed andor changed individually by specifying the row andor column number of the element In a row or column vector only a single number is required In a matrix both the row and column number must be speci ed with A i j choosing the ith row and jth column gtgt y3 ans 2 gtgt A32 ans 8 However you must specify a position within the matrix or vector Otherwise MATLAB will complain gtgt A23 Index exceeds matrix dimensions You may change individual elements of a matrix or vector in this way gtgt y3 y3 2 y A O 2 3 7 12 5 You may also select parts of a matrix or vector using B 2 12 to specify the rst two elements ofthe second row ofB or B 2 for the entire second row of B gtgt B2 ans 2 7 8 Often you need to access elements at or near the end of a vector You can do this by typing y end or y 2 end or y 2 end 1 This is very handy especially when you want to plot part ofa vector Getting information about variables To see the size ofa variable or its length number of rows type size A or length A To see all of the variables that are currently in the MATLAB workspace type whos gtgt whos Name Size Bytes Class A 3x2 48 double array B 2x3 48 double array ans 1x3 24 double array t 1x7 56 double array u 1x7 56 double array w 1x5 40 double array x 1x1 8 double array y 1x5 40 double array z 5x1 40 double array Grand total is 45 elements using 360 bytes Elementary computations Variables may be added subtracted multiplied and divided as long as the rules of arithmetic and linear algebra are obeyed ie you can t divide by zero or multiply a 1x2 matrix by a 3x4 matrix Multiplying a vector or matrix by a scalar will scale each element of the vector or matrix by the value of the scalar gtgtC212 34 gtgtV21234 2 4 6 8 Adding a scalar to a vector or matrix will add the value of the scalar to each element of the matrix gtgtD212 34 gtgt s 2 05 2 3 4 5 6 7 Matrices and vectors may be added or subtracted as long as they are the same size They may be multiplied as long as there are the same number of columns in the rst as there are rows in the second gtgt s 2Oz5 ans 2 5 8 11 14 17 gtgt AB ans 4 14 16 14 58 92 10 71 184 gtgt BA ans 153 81 117 93 Putting a period in front of the multiplication diVision or power operator performs that operation componentwise ie x y x y or x 2 xtz gtgt 1 2 3 4 2 1 4 9 16 For loops To run a command more than once as an index varies you may use a for loop In the following example a for loop is used to compute n for n l 10 gtgt j 1 gtgt for i 110 339 ji end You may also do the loop with the values of i in the matrix 0 2 3 6 gtgt for i O 2 3 M DO SOMETHING end Note that MATLAB will not run the for loop until you have hit return after typing end to indicate the end of the for loop I nd it useful to use the tab key to indent all commands within the loop This can make the code much easier to read and understand If statements MATLAB uses a similar structure for if statements gtgt if i DO SOMETHING elseif i DO SOMETHING ELSE elseif i DO SOMETHING ELSE else DO SOMETHING IF i gt 4 or i lt 0 end Plotting To make a line plot of t versus simply type gtgt plot tsqrt t To make a plot to t versus and t versus t on the same plot type gtgt plot t sqrt t t t putting each xy pair together You can add symbols or use symbols instead of lines by adding commands to each pair gtgt plottsqrtt tt O39 See help plot for more information on plotting and a catalog of the available symbols and line types Use legend sqrt t t to label the di erent lines Title your plot by typing title A plot of t versus sqrt t You can add labels to the axes similarly xlabel t or ylabel sqrt t Other useful commands axis plot3 and for a bit of fun comet 6 Running m les You can write scripts of MATLAB functions which may be run as a whole Save the les with a m extension so that MATLAB will recognize it as a script Type gtgt junk to run the script in the le junkm DO NOT USE THE NAME OF AN INTRINSIC MATLAB FUNCTION AS THE TITLE OF YOUR SCRIPT Creating your own functions You can create your own functions to use in MATLAB There are two options for this If your function is simple ie f x x2 2x then you may enter it using the command inl ine gtgt f inline x 2 2x f Inline function fx x 2 2x We can then apply this function to a scalar or vector since we used the 2 command for computing the square of X gtgt f05 ans O 3 8 15 24 35 If the function is more complicated you may create a le whose name ends in m which tells MATLAB how to compute the function Type help function to see the format of the function le Our function here could be computed using the following le called f m function output fx output x 2 2x This function is called from MATLAB in the same way as above ie f x where x can be a scalar or vector Impmting data Use the load command to bring data into matlab from an external le gtgt load filename

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "Selling my MCAT study guides and notes has been a great source of side revenue while I'm in school. Some months I'm making over $500! Plus, it makes me happy knowing that I'm helping future med students with their MCAT."

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.