### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# lec 19

RPI

### View Full Document

## 10

## 2

1 review

## Popular in

## Popular in Department

This 5 page Class Notes was uploaded by Charles Okaeme on Sunday February 28, 2016. The Class Notes belongs to at Rensselaer Polytechnic Institute taught by in Fall 2015. Since its upload, it has received 10 views.

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 02/28/16

MANE 6600/ ECSE 6400: Systems Analysis Techniques Fall 2015 Lecture 19 | November 16 Lecturer: S. Mishra Scribe: Optimal Control (LQR) Overview 1. Optimal Control for Discrete Time 2. Dynamic Programming Approach 3. Bellman’s Optimality Principle 4. Steady-State LQR 19.1 Optimal Control for Discrete Time 19.1.1 Reminders from Last Lecture For continuous time: u(t) = K(x ▯ x(t)) drives x(t) to x . d d For discrete time: u(k) = K(d = x(t)) drives x(k) td x . Place poles in LHP for continuous time, and place poles in unit circle for discrete time. In other words, design K such that specf(A ▯ BK)g 2 unit circle for discrete time. Note that placing poles closer to the origin will create a faster reaction, while placing poles closer to the edge of the circle will create a more oscillatory response. 19.1.2 Discrete Time Cost Function We want to choose K to minimize an objective (or cost) function. Given a discrete time system given by x(k + 1) = Ax(k) + Bu(k) (19.1) y(k) = Cx(k) (19.2) x(0) = x0 (19.3) 19-1 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 we can write the cost function for an experiment lasting from k = 0 to k = N as follows: X▯1 J = S(x(N)) + (L(x(k);u(k);k) (19.4) k=0 P N▯1 where S(x(N)) is the terminal cost andk=0(L(x(k);u(k);k) is the running cost. 19.1.3 Example Given N▯1 T X T T J = x(N) Sx(N) + (u(k) Ru(k) + x(k) Qx(k)) (19.5) k=0 we can write the following: N▯1 2 X 2 2 J = kx(N)k + (kx(k)k + ku(k)k ): (19.6) k=0 This brings us to the question of how to design u, where u = u(0);u(1);:::;u(N). 19.1.4 Optimal Control Constraints: u(k) 2 U, where U is the set of inputs within the constraint range. ▯ ▯ ▯ ▯ ▯ Let U = fu(0) ;u(1) ;:::;u(N ▯1) gbe the optimal control sequence and J be the optimal cost. We can write ▯ J = min J (19.7) u(k)2U ▯ ) u = argminJ: (19.8) u(k)2U 19.2 Dynamic Programming Approach There are many ways to solve the optimization problem, but we will explore dynamic pro- gramming. Assume that we already know u(0) ;:::;u(k ▯ 1) (i.e., this has been determined from some other method). The remaining running cost can be designatedkby u = fu(k);:::;u(N ▯1)g. The cost to go from x(k) to x(N) is designated as J (x(k)). This relationship is illustrated k below. 19-2 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 We can write Jk(x(k)) as X▯1 J (x(k)) = minfx(N) Sx(N) + (L(x(i);u(i);i))g (19.9) k k i=k N▯1 T X = min min fx(N) Sx(N) + (L(x(i);u(i);i))g (19.10) k fk+1;:N▯1g i=k X▯1 = minminfx(N) Sx(N) + (L(x(i);u(i);i))g (19.11) uk Uk+1 i=k N▯1 T X = minfL(x(k);u(k);k) + min(x(N) Sx(N) + (L(x(i);u(i);i)))g (19.12) u(k) Uk+1 i=k ▯ = minfL(x(k);u(k);k) + Jk+1(x(k + 1))g (19.13) u(k) 19.2.1 Bellman’s Optimality Principle An optimal control sequence has the property that regardless of the initial control input and initial state, the remainder of the control input is an optimal sequence for the rest of the way (if you begin with the optimal trajectory, then the rest of the way is also optimal). Note that since we are still on the optimal track for the rest of the way, this means we can keep constructing the input from in▯nity backwards. Dynamic Programming (DP) is a recursive formula in reverse time. Let x(k + 1) = Ax(k) + Bu(k) ▯ T JN(x(N)) = x (N)Sx(N) (19.14) X▯1 1 T 1 ▯ T T ▯ J = x (N)Sx(N) + x (k)Qx(k) + u (k)Ru(k) (19.15) 2 2 k=0 19-3 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 Using DP ▯ ▯ ▯ 1 T 1 T ▯ J k(x(k)) = min(k) x (k)Qx(k) + u (k)Ru(k) + J k+1(x(k + 1)) (19.16) u 2 2 1 J (x(N)) = x (N)Sx(N) initialization (19.17) N 2 Note: We use 1/2 as a coe▯cient in the quadratic for ease of di▯erentiation. @ N ▯ 1, 1 1 1 J▯ (x(N ▯ 1)) = min f x (N ▯1)Qx(N ▯1)+ u (N ▯1)Ru(N ▯1)+ x (N)Sx(N)g T (N▯1) u(N▯1) 2 2 2 (19.18) We know x(N) = Ax(N ▯ 1) + Bu(N ▯ 1), so we can plug it in. Also, x(N ▯ 1) does not depend on u(N ▯ 1). Solving (19.18), we get ▯ ▯ u▯ = ▯ R + B SB ▯1B SAx(N ▯ 1) state feedback law (19.19) N▯1 | {z } K Find J ▯ (x(N ▯ 1)) = 1x (N ▯ 1)P(N ▯ 1)x (N ▯ 1) where (N▯1)h 2 i T T ▯ T ▯▯1 T P(N ▯1) = Q + A SA ▯ A SB R + B SB B SA . This expression does not depend on x(N). Further; T T ▯ T ▯▯1 T P(i ▯ 1) = A P(i)A + Q ▯ A P(i)B R + B P(i)B B P(i)A (19.20) ▯ T ▯▯1 T u(i) = ▯ R + B P(i)B B P(i)Ax(i) (19.21) u(i) is a state feedback with a time varying gain. This linear control law is called linear quadratic regulator (LQR). It is quite computationally expensive, but there is a better way known as steady-state LQR. 19.2.2 Steady-State LQR 1. Let (A;B) be controllable 2. Let N ) 1 (i.e. the experiment is very long) 3. Claim: P(i) ) P ss as i ) 1 We previously had a backward evolution of P, but now we assume that P settles to some steady state. Therefore, P(i ▯ 1) = P(i) = Pss ▯ ▯▯1 P ss= A P Ass Q ▯ A P B R ssB PB T B P Ass (19.22) 19-4 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 This is the Algebraic Riccati Equation (ARE). P ss will exist if (A;B) is controllable and R > 0. R > 0 means that we cannot generate an in▯nite amount of control e▯ort, so the penalty on control e▯ort is non-zero. ▯ T ▯▯1 T K = R + B P B ss B P ss (19.23) u = ▯Kx (19.24) We can solve P sso▯ine. All other values are known, so K is a constant gain. Q is the weighting on the control e▯ort and is usually chosen as a diagonal matrix (with the weighting elements on the diagonal). Q must also be positive de▯nite. Robustness can be increased by increasing R, while tracking is improved by increasing Q. Apart from PID control, LQR is the most commonly used control technique. 19-5

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "I used the money I made selling my notes & study guides to pay for spring break in Olympia, Washington...which was Sweet!"

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.