Class Note for STAT 635 at OSU 02
Class Note for STAT 635 at OSU 02
Popular in Course
Popular in Department
This 20 page Class Notes was uploaded by an elite notetaker on Friday February 6, 2015. The Class Notes belongs to a course at Ohio State University taught by a professor in Fall. Since its upload, it has received 22 views.
Reviews for Class Note for STAT 635 at OSU 02
Report this Material
What is Karma?
Karma is the currency of StudySoup.
Date Created: 02/06/15
STATISTICS 635 SUMMER 2005 INTRODUCTION TO ARMA MODELS If you do not learn and master the art of unconditional contentment on Earth here now you will not be able to notice let alone experience the universally promised everlasting peace and joy of Heaven The Eternal N ower DEFINITION OF ARMApq PROCESSES De nition 1 Xt is an ARMAp q process ifXt is stationary and if for every t Xt 1Xt1 th Z 61Zt1 QQZH 1 where Z N WNO 02 and the polynomials Zgt1 IZ p2p 2 and 6216126qzq 3 have no common factors EH The process Xt IS said to be an ARMAp q process with mean u If Xt n IS an ARMAp q process STATISTICAL ANALYSIS OF TIME SERIES 1 STATISTICS 635 SUMMER 2005 EH lf the polynomials 2 and 3 have no common factors7 then the model cannot be reduced to a simpler one EH lf the polynomials 2 and 3 do have common factors7 then there are redundant parameters that unnecessarily complicate the analysis EH As we did for the ARMA1 1 model7 we will use the backward shift operator B with Bth Xtij and Bth Ztjj Ol1l2 to rede ne the ARMAQ q process of 1 more concisely as BgtXt 9ltBgtZt 4 D is known as the regressive operator7 and is a pth degree polynomial in B of the form 27 ie ltBgt1 IB po D 6B is known as the moving average operator7 and is a qth degree polynomial in B of the form 37 ie 63 1613quot396qu D Clearly7 if 1 then ARMAQ q reduces to lVIAltqgt7 but the notation ARMAO q is sometimes used to stress the fact that MA is a member of the ARMA family D Similarly7 if 6B 1 then ARMAQ q reduces to ARltpgt7 but the notation ARMAQ O is sometimes used to stress the fact that AR is a member of the ARMA family STATISTICAL ANALYSIS OF TIME SERIES 2 STATISTICS 635 SUMMER 2005 EH Recall our study of the ARMA1 1 model Xt gz5Xt1 Z 6Z4 for every t D ARMA1 1 is stationary if y 17 ie 1 gb y O or 1gz5 07 which is equivalent gbz1 gz527 0 for Z1 D ARMA1 1 is causal if lt 17 which translates into gbz1 gz5z0 for Z gt1 OIquot gz5z1 gz527 0 for Z 1 CAUSALITY OF THE ARMAQU q PROCESS Proposition 1 A stationary solution Xt of equation 1 exists and is also the unique stationary solution if and only if gbz1 gblZ gbpzp7 0 foraIZ1 5 De nition 2 An ARMAp q process Xt is causal or a causal function of Z if there exist absolutely summable constants 220 lt 00 such that Xt Zr thj for all t 6 j0 which is equivalent 5ltZgt1 51Z pzp7 0 forallz 1 lt7 STATISTICAL ANALYSIS OF TIME SERIES 3 STATISTICS 635 SUMMER 2005 IMPORTANT REMARKS EH For ARMA1 17 it is relatively easy straightforward to see the relationship between the causality condition lt 1 and the root Z i of the polynomial 1 gbz EH For ARMAQ q7 it is more di icult to study the relationship be tween the values of the parameters gbl gbg gbp and the roots of the polynomial 1 gblz 5222 quzp EXAMPLE CAUSALITY OF AR2 EH The autoregressive of order 27 ie ARlt2gt has equation 1 92513 2BQXI Zt EH ARlt2gt is causal if the roots of the polynomial 5ltZgt 1 1Z 2Z2 lie outside the unit circle Z 1 Which nIeans7 Cbi i 9252 49252 2ltZ52 EH Let 1 and 2 denote the two roots These two roots may be gt1 D Real and distinct D Real and equal B Complex conjugate pair STATISTICAL ANALYSIS OF TIME SERIES 4 STATISTICS 635 SUMMER 2005 EH With 51 and 52 thus ole nte7 we can write ltZgtlt1 f1Zgt1 51Zgt EH Then the ARlt2gt equation can be written as 1 143 51BgtXt Zt EH Or equivalently7 for 1 gt 1751 75 2 1 lt51 51 lt 1 1gtBQ XI ZI EH We therefore derive lt51 ill 5f1 2 1524 BS The conditions of the causality of AR2 are ltZ52l lt 1 lt51 9252 lt 1 1 9252 lt 1 8 EH lt is interesting to note that the conditions of causality are the same for both ARltpgt and ARMAQ q EH For a causal ARMAQ q7 we can write 93 9253 Xt Zt which implies STATISTICAL ANALYSIS OF TIME SERIES 5 STATISTICS 635 SUMMER 2005 FINDING THE COEFFICIENTS 1 OF AN ARMAp q EH Writing 63 in polynomial fornI gives 1 qu pzpw0wlzm161zm6qzq Equating the coe icients Z7 for j O 1 2 gives 1 we 91 1 09251 92 2 19251 0952 19 9739 W7 Z skwjek k1 Clearly7 60 1 and Q7 O for j gt q7 and nally 1 O for j lt O EXERCISE 1 Consider the ARMAQ 1 process Xt 08Xt71 01Xt2 Zt 03Zt1 EH ls this process causal EXERCISE 2 Consider the ARlt2gt process Xt 07Xt1 01Xt2 Zt EH ls this process causal EH Find the coef cients 1 of its linear process representation STATISTICAL ANALYSIS OF TIME SERIES 6 STATISTICS 635 SUMMER 2005 INVERTIBILITY OF THE ARMAp q PROCESS EH invertibility is related to the MA part of the ARMA nIodel7 and depends on the behavior of the qth degree polynomial 6B EH it consists of deriving an in nite autoregressive representation7 ie 1 00 r 00 2t 6ltBgt ltBgtXt 7rltBgtXI 79Bth Tthej Where no 1 and 220 7rj lt 00 De nition 3 An ARMAQ q process Xt is invertible if there exist absolutely summable constants 79 220 7rj lt 00 such that Z Z antj for all t 9 j0 which is equivalent 6216126qzq0 for all Z 1 10 EB The coef cients 79 can be determined by solving where 00 7IZ Z 7rjzj j0 EH More specifically7 q wj26k7rjk qu j0I 11 k l Wheregbo 17gbj0forjgtp7and7r00forjlt0 STATISTICAL ANALYSIS OF TIME SERIES 7 STATISTICS 635 SUMMER 2005 EXERCISE 3 Consider the following process Xt 01Xt1 012Xt2 Z 09Zt1 02Zt2 EH ls this really an ARMAQ 2 process Justify your answer EH ls this process causal EH lf so7 nd the coef cients 1 of its linear process representation EH ls this process invertible EH lf so7 nd the coef cients 7g of its linear process representation EXERCISE 4 Consider the following ARMAQ 1 process Xt 075Xt1 05625Xt2 Zt125Zt1 EH ls this really an ARMAQ 1 process Justify your answer EH ls this process causal EH lf so7 nd the coef cients 1 of its linear process representation EH ls this process invertible EH lf so7 nd the coef cients 7g of its linear process representation EXERCISE 5 Consider the following ARMAQ 2 process Xt 04Xt1 045Xt2 Z Zt1 025Zt2 EH ls this really an ARMAQ 1 process Justify your answer EH ls this process causal EH ls this process invertible STATISTICAL ANALYSIS OF TIME SERIES 8 STATISTICS 635 SUMMER 2005 ACF AND PACF OF ARMApq EH We rst express the equation of the ARMA process as a linear process7 and then use the proposition that gives the general formula for computing the ACVF of linear processes EH First we determine the ACVF of the causal ARMAQ q ie XI 2 MM j0 where w m 1 EH First method From a previous proposition7 j0 Wt EIXAIXII 0 2 Z ijj hl j0 EXERCISE 6 Derive the ACVF of an ARMA1 1 process Xt ngFl Z 6amp4 Z N WNO 02 EB First nd 70 EH Express yh as a function of yh 1 for h gt 1 EXERCISE 7 Derive the ACVF of an MAq process Xt Z 61ZH QQZH Z N WN0 02 EB Comment of the distinct feature of this ACVF STATISTICAL ANALYSIS OF TIME SERIES 9 STATISTICS 635 SUMMER 2005 FINDING THE ACF OF A CAUSAL ARMA11 EH For a linear process given by representation Xt ijz r Vt j0 EH The ACVF at lag h is given by Wt 02 Z ijj hl39 j0 Remember that 0 1 and 1 gbj l b 9 for j gt 0 EB Exercise D Show that 70 02 and 71 02 11g2662 6 D Establish that vlthgt mlth 1gt0 h21 D Finally deduce that lt1 6M 6 The ACF for an ARMA1 1 process is therefore plthgt W h 21 12 lcb h21 STATISTICAL ANALYSIS OF TIME SERIES 10 STATISTICS 635 SUMMER 2005 FINDING THE ACF OF AN ARMAp q PROCESS EH Reconsider the ARMAQ q process Xt 1Xt71 Cprpp Zt 61Zt71quot qutiq EH Multiply both sides of the equation by Xtk and take expectations The result is the general homogeneous difference equation for 70L given by W Wltk 1 IMO pgt 0 k 2 m 13 EB With initial conditions 00 W 5170 1gt IMO p 02 2 6mm 14 j0 forOg k ltm7wheremn1axpq17ij0forj lt07 601 and Q7 Oforj 65 O12 q EH The solution of the homogeneous equation 14 is known to be M 045141 one ap gh h 2 m p 15gt Where 5152 fp are the roots of O and are assumed to be distinct a1 a2 up are arbitrary constants EH To complete the determination of 70L we need to also satisfy 13 This is done by substituting 15 into 137 thereby producing m linear equations that uniquely deternIine D The constants a1 a2 Ozp D The m p autocovariances 70L for 0 g h lt m p STATISTICAL ANALYSIS OF TIME SERIES 11 STATISTICS 635 SUMMER 2005 EXAMPLES OF ACVF AND ACF DETERMINATION EXAMPLE 1 ACVF AND ACF OF AN ARMA1 1 PROCESS EH Show that for an ARMA11 process 10 vlt1gt02lt1 002gt 16 md 11 92570 020 17 EB Show that WI 5701 1 0 h gt1 EH Deduce that the solution IS of the form VUL 042 h gt1 for some constant oz EH Solve for 70 and 717 to obtain 1 26gb 62 i 2 i a 1 2 and lt gtlt gt 1 6 gb 6 11 02 11 21 and nally 02 gbhil h 21 1 gb The ACF for an ARMA1 1 process is therefore lt1 6gtlt 6gt gt pm 126gb62 h l lt18 STATISTICAL ANALYSIS OF TIME SERIES 12 STATISTICS 635 SUMMER 2005 EXAMPLE OF ACF DETERMINATION FOR AR2 EH Recall that we rewrote our causal ARlt2gt as 1 lt51 I1gtB lt 1 1gtBQXI Zt EH We therefore derive cbl f1 f1 lt52 5f1 1 EH By letting 1 T626 and 2 T6497 with O lt 6 lt 7L we get 02r4r h sinh6 w h ll l r2 1r4 27quot cos 26 sIrI6 where 2 1 tanw 2 l taIIQ r 1 EH lII the plots that follow we explore various oases7 namely D51 2 and 2 57 so that b gbl 07 and 2 O1 D 51 109 and 2 27 so that b gbl 14 and 2 O45 D51 109 and 2 27 so that b gbl O4 and 2 045 D51 21i 3 and 52 21 i gt37 so that b gbl 06 and 2 O45 STATISTICAL ANALYSIS OF TIME SERIES 13 STATISTICS 635 SUMMER 2005 PLOTTING THE ACF FOR DIFFERENT AR2 PROCESSES ACF ACF EH Provide some interesting comments about the above ACF plots In 08 Ga 04 02 an In 05 an ACF ofAR2 with Xi 1 2 and Xi 2 5 ACF of AR2 with xi 1 10I9 and Xi 2 2 STATISTICAL ANALYSIS OF TIME SERIES ACF ACF In 05 an 405 02 04 Ga 08 In an ACF of AR2with xi 1 109 and Xi 2 2 ACF of AR2with xi 1 21isqrt3l3 and xi 2 21isqrt3l3 14 STATISTICS 635 SUMMER 2005 FINDING THE PACF OF AN ARMAp q PROCESS De nition 4 The partial autocorrelation function PACF ofan ARMA process Xt is the function a de ned by the equations MO 1 and 0400 Cbhh h 21 Where gbhh is the last component of the vector h 17171 19 With matrix Fm 7i j j1 and vector 7 71 72 7h THE SAMPLE PACF EH Given observations X1X2 Xn with X y x7 for some i and j EH The sample PACF h is given by 640 1 and 07W Gigm h 21 Where ghhh is the last component of the vector 11 331 20 STATISTICAL ANALYSIS OF TIME SERIES 15 STATISTICS 635 SUMMER 2005 ANOTHER DEFINITION OF THE PACF OF Xt De nition 5 The partial autocorrelation function PACF ofa zero mean stationary time series Xt is de ned as 2511 corrX1Xoplt1gt Cbhh C0rrXh 71001 X0 fh71ltX0gtgt 21 Where fh71ltXhgt fltXt1lllXtl1gt PltXhXh71gt 39 quot 7X1 minimizes the mean squared linear prediction error E Xh PXhXh1 X12 EH Consider all the variables Xt1 Xth17 and the linear function fltXt71quot 39 aXt7h1gt 61Xt7139 hilXtiHl EH If the linear effect of each variable is removed7 then by stationarity7 Cbhh is the correlation between variables Xt and Xth EH We now explain Why this de nition helps justify the need for another correlation measure in our effort to identify time series models Remark It should become quickly clear to you that analytical com putations of PACF though doable for AR processes7 is indeed very complicated for ARMA processes Hence the great appeal of iterating algorithms along the lines of the Durbin Levinson STATISTICAL ANALYSIS OF TIME SERIES 16 STATISTICS 635 SUMMER 2005 WHY DO WE NEED THE PACF Consider a causal ARC process with its well known equation Xt CbXtil Zt EH lt is easy to see that 72 52 VO7 so that the autocorrelation plt2gt is not zero This is because Xt depends on Xt72 through Xt1 EH The chain of dependence can be broken by removing the effect of Xt1 from Xt72 and Xt Xt CbXtil and XtiQ CbXtil EH Instead of computing covX1tXt27 compute COVltXt CbXtila Xti2 CbXtil EH Clearly7 COVltXt qutilaXtiQ CbXtil COVltZta Xti2 CbXtil 0 EH For the correlation between Xt and Xt737 break the dependence by removing the effects of Xt1 and Xt72 to obtain zero correlation Xt Xvi Xt72gt and Xe XVI Xt72gt EH More generally7 get zero correlation between Xt and leh7 by using Xt fltXt71gtXt72a XHHI and Xtih fltXt717 Xt72gt 39 quot aXtih1gt STATISTICAL ANALYSIS OF TIME SERIES 17 STATISTICS 635 SUMMER 2005 EXAMPLE 1 DERIVE THE PACF FOR AR1 PROCESSES EH First provide the theoretical derivation EH Simulate ARC processes for gb 09 O9 05 O5 EH Plot the series EH Plot the corresponding ACF and PACF Compare and Comment AR1 with phi 09 AR1 with phi o9 T e T m i 7 i i i i i i i i i i El 5D TDD 15D ZEIEI El 5D TDD 15D ZEIEI Time Time AR1 with phi 05 AR1 with phi o5 Equot T tn E T tn T T T i i i i i T i i i i i El 5D TDD 15D ZEIEI El 5D TDD 15D ZEIEI Time Time STATISTICAL ANALYSIS OF TIME SERIES 18 STATISTICS 635 W In W m on nn n2 no nn nn n2 nn n2 no nn Inn on on ACF furAR1 Wm pmzu 5 Paa SUMMER 2005 FACF AR1 Wm pmzu 5 Figure 1 ACF and PACF of an AR1 process X 09xt1 Z ACF furAR1 m 5595 5 Fala W FACF AR1 m 5595 5 Figure 2 ACF and PACF of an AR1 process X 709xt51 Z ACF furAR1 Wm pmzu 5 innnnihn 393939 NH VVVVVVVVVVVVVVVVVVVVVVVVV ACF furAR1 m 5595 5 Palm Fawn no nn no n2 n no n5 n5 Ano n3 n2 no nn no FACF AR1 Wm pmzu 5 Figure 4 ACF and PACF of an AR1 process X 705xt51 Z STATISTICAL ANALYSIS OF TIME SERIES 19 STATISTICS 635 SUMMER 2005 EXAMPLE 2 DERIVE THE PACF FOR ARp PROCESSES AR2with phi 1 07 and phi 2 01 ts D an mu 15D ZEIEI Time Figure 5 Plot of an AR2 process X 07xt1 7 01xt2 Zt AR2 with phi 1 07 and phi 2 01 AR2with phi 1 07 and phi 2 01 Ga 1U 04 00 08 i i 04 ACF Famai ACF U 2 1 ti INN Wiiii iiWHi IIIIIWIHH W UU HH W WH J i i i i i i i i i i D In 4D EEI ED mu 5 1D 15 2D an an 402 i Lag Lag Figure 6 ACF and PACF of an AR2 process X 07xt1 7 01xt2 Zt EH Describe the difference between the two correlation measures STATISTICAL ANALYSIS OF TIME SERIES 20