×

### Let's log you in.

or

Don't have a StudySoup account? Create one here!

×

or

## lec 19

1 review
by: Charles Okaeme

10

2

5

# lec 19

Charles Okaeme
RPI

Get a free preview of these Notes, just enter your email below.

×
Unlock Preview

LQR
COURSE
PROF.
TYPE
Class Notes
PAGES
5
WORDS
CONCEPTS
LQR
KARMA
25 ?

## 2

1 review
"No all-nighter needed with these notes...Thank you!!!"
Tre Skiles

## Popular in Department

This 5 page Class Notes was uploaded by Charles Okaeme on Sunday February 28, 2016. The Class Notes belongs to at Rensselaer Polytechnic Institute taught by in Fall 2015. Since its upload, it has received 10 views.

×

## Reviews for lec 19

No all-nighter needed with these notes...Thank you!!!

-Tre Skiles

×

×

### What is Karma?

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 02/28/16
MANE 6600/ ECSE 6400: Systems Analysis Techniques Fall 2015 Lecture 19 | November 16 Lecturer: S. Mishra Scribe: Optimal Control (LQR) Overview 1. Optimal Control for Discrete Time 2. Dynamic Programming Approach 3. Bellman’s Optimality Principle 4. Steady-State LQR 19.1 Optimal Control for Discrete Time 19.1.1 Reminders from Last Lecture For continuous time: u(t) = K(x ▯ x(t)) drives x(t) to x . d d For discrete time: u(k) = K(d = x(t)) drives x(k) td x . Place poles in LHP for continuous time, and place poles in unit circle for discrete time. In other words, design K such that specf(A ▯ BK)g 2 unit circle for discrete time. Note that placing poles closer to the origin will create a faster reaction, while placing poles closer to the edge of the circle will create a more oscillatory response. 19.1.2 Discrete Time Cost Function We want to choose K to minimize an objective (or cost) function. Given a discrete time system given by x(k + 1) = Ax(k) + Bu(k) (19.1) y(k) = Cx(k) (19.2) x(0) = x0 (19.3) 19-1 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 we can write the cost function for an experiment lasting from k = 0 to k = N as follows: X▯1 J = S(x(N)) + (L(x(k);u(k);k) (19.4) k=0 P N▯1 where S(x(N)) is the terminal cost andk=0(L(x(k);u(k);k) is the running cost. 19.1.3 Example Given N▯1 T X T T J = x(N) Sx(N) + (u(k) Ru(k) + x(k) Qx(k)) (19.5) k=0 we can write the following: N▯1 2 X 2 2 J = kx(N)k + (kx(k)k + ku(k)k ): (19.6) k=0 This brings us to the question of how to design u, where u = u(0);u(1);:::;u(N). 19.1.4 Optimal Control Constraints: u(k) 2 U, where U is the set of inputs within the constraint range. ▯ ▯ ▯ ▯ ▯ Let U = fu(0) ;u(1) ;:::;u(N ▯1) gbe the optimal control sequence and J be the optimal cost. We can write ▯ J = min J (19.7) u(k)2U ▯ ) u = argminJ: (19.8) u(k)2U 19.2 Dynamic Programming Approach There are many ways to solve the optimization problem, but we will explore dynamic pro- gramming. Assume that we already know u(0) ;:::;u(k ▯ 1) (i.e., this has been determined from some other method). The remaining running cost can be designatedkby u = fu(k);:::;u(N ▯1)g. The cost to go from x(k) to x(N) is designated as J (x(k)). This relationship is illustrated k below. 19-2 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 We can write Jk(x(k)) as X▯1 J (x(k)) = minfx(N) Sx(N) + (L(x(i);u(i);i))g (19.9) k k i=k N▯1 T X = min min fx(N) Sx(N) + (L(x(i);u(i);i))g (19.10) k fk+1;:N▯1g i=k X▯1 = minminfx(N) Sx(N) + (L(x(i);u(i);i))g (19.11) uk Uk+1 i=k N▯1 T X = minfL(x(k);u(k);k) + min(x(N) Sx(N) + (L(x(i);u(i);i)))g (19.12) u(k) Uk+1 i=k ▯ = minfL(x(k);u(k);k) + Jk+1(x(k + 1))g (19.13) u(k) 19.2.1 Bellman’s Optimality Principle An optimal control sequence has the property that regardless of the initial control input and initial state, the remainder of the control input is an optimal sequence for the rest of the way (if you begin with the optimal trajectory, then the rest of the way is also optimal). Note that since we are still on the optimal track for the rest of the way, this means we can keep constructing the input from in▯nity backwards. Dynamic Programming (DP) is a recursive formula in reverse time. Let x(k + 1) = Ax(k) + Bu(k) ▯ T JN(x(N)) = x (N)Sx(N) (19.14) X▯1 1 T 1 ▯ T T ▯ J = x (N)Sx(N) + x (k)Qx(k) + u (k)Ru(k) (19.15) 2 2 k=0 19-3 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 Using DP ▯ ▯ ▯ 1 T 1 T ▯ J k(x(k)) = min(k) x (k)Qx(k) + u (k)Ru(k) + J k+1(x(k + 1)) (19.16) u 2 2 1 J (x(N)) = x (N)Sx(N) initialization (19.17) N 2 Note: We use 1/2 as a coe▯cient in the quadratic for ease of di▯erentiation. @ N ▯ 1, 1 1 1 J▯ (x(N ▯ 1)) = min f x (N ▯1)Qx(N ▯1)+ u (N ▯1)Ru(N ▯1)+ x (N)Sx(N)g T (N▯1) u(N▯1) 2 2 2 (19.18) We know x(N) = Ax(N ▯ 1) + Bu(N ▯ 1), so we can plug it in. Also, x(N ▯ 1) does not depend on u(N ▯ 1). Solving (19.18), we get ▯ ▯ u▯ = ▯ R + B SB ▯1B SAx(N ▯ 1) state feedback law (19.19) N▯1 | {z } K Find J ▯ (x(N ▯ 1)) = 1x (N ▯ 1)P(N ▯ 1)x (N ▯ 1) where (N▯1)h 2 i T T ▯ T ▯▯1 T P(N ▯1) = Q + A SA ▯ A SB R + B SB B SA . This expression does not depend on x(N). Further; T T ▯ T ▯▯1 T P(i ▯ 1) = A P(i)A + Q ▯ A P(i)B R + B P(i)B B P(i)A (19.20) ▯ T ▯▯1 T u(i) = ▯ R + B P(i)B B P(i)Ax(i) (19.21) u(i) is a state feedback with a time varying gain. This linear control law is called linear quadratic regulator (LQR). It is quite computationally expensive, but there is a better way known as steady-state LQR. 19.2.2 Steady-State LQR 1. Let (A;B) be controllable 2. Let N ) 1 (i.e. the experiment is very long) 3. Claim: P(i) ) P ss as i ) 1 We previously had a backward evolution of P, but now we assume that P settles to some steady state. Therefore, P(i ▯ 1) = P(i) = Pss ▯ ▯▯1 P ss= A P Ass Q ▯ A P B R ssB PB T B P Ass (19.22) 19-4 MANE 6600/ ECSE 6400Lecture 19 | November 16 Fall 2015 This is the Algebraic Riccati Equation (ARE). P ss will exist if (A;B) is controllable and R > 0. R > 0 means that we cannot generate an in▯nite amount of control e▯ort, so the penalty on control e▯ort is non-zero. ▯ T ▯▯1 T K = R + B P B ss B P ss (19.23) u = ▯Kx (19.24) We can solve P sso▯ine. All other values are known, so K is a constant gain. Q is the weighting on the control e▯ort and is usually chosen as a diagonal matrix (with the weighting elements on the diagonal). Q must also be positive de▯nite. Robustness can be increased by increasing R, while tracking is improved by increasing Q. Apart from PID control, LQR is the most commonly used control technique. 19-5

×

×

### BOOM! Enjoy Your Free Notes!

×

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

Bentley McCaw University of Florida

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

Janice Dongeun University of Washington

#### "I used the money I made selling my notes & study guides to pay for spring break in Olympia, Washington...which was Sweet!"

Jim McGreen Ohio University

Forbes

#### "Their 'Elite Notetakers' are making over \$1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!
×

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com