Time Series Analysis - 時間序列模型基本概念：AR, MA, ARMA, ARIMA 模型

時間序列模型，AR、MA、ARMA、ARIMA模型等基礎知識，在訊號、金融時序分析通用，頗有萬物歸一之感^^

一、自迴歸模型 (AR, Autoregressive Models)

自身迴歸用同一變數例如x的之前各期，亦即 $x_{1}$ 至 $x_{t-1}$ 來預測本期 $x_{t}$ 的表現，並假設它們為一線性關係。

1. 自迴歸模型定義

The notation AR(p) indicates an autoregressive model of order p. The AR(p) model is defined as

$X_t = c + \sum_{i=1}^p \varphi_i X_{t-i}+ \varepsilon_t \,$

where $\varphi_1, \ldots, \varphi_p$ are the parameters of the model, $c$ is a constant, and $\varepsilon_t$ is white noise.

[用心去感覺] X的當期值等於一個或數個落後期的線性組合，加常數項，加隨機誤差。

2. AR(p)探討

AR(0) : the simplest one, which has no dependence between the terms. Only the noise term contributes to the output of the process, so AR(0) corresponds to white noise.

AR(1) : AR(1) process with a positive $\varphi$, only the previous term in the process and the noise term contribute to the output.

• If $\varphi$ is close to 0, then the process still looks like white noise
• but as $\varphi$ approaches 1, the output gets a larger contribution from the previous term relative to the noise. This results in a "smoothing" or integration of the output, similar to a low pass filter.

AR(2) : the previous two terms and the noise term contribute to the output.

•    If both $\varphi_1$ and $\varphi_2$ are positive, the output will resemble a low pass filter, with the high frequency part of the noise decreased. 
•    If $\varphi_1$ is positive while $\varphi_2$ is negative, then the process favors changes in sign between terms of the process. The output oscillates. This can be likened to edge detection or detection of change in direction.

3. Example: An AR(1) process

An AR(1) process is given by : $X_t = c + \varphi X_{t-1}+\varepsilon_t\,$

The process is wide-sense stationary if $\varphi$ less than  1 since it is obtained as the output of a stable filter whose input is white noise.

If the mean is denoted by $\mu$, it follows from : 

$\operatorname{E} (X_t)=\operatorname{E} (c)+\varphi\operatorname{E} (X_{t-1})+\operatorname{E}(\varepsilon_t),$

$\mu=c+\varphi\mu+0,$

$\mu=\frac{c}{1-\varphi}$

The variance is : 

$\textrm{var}(X_t)=\operatorname{E}(X_t^2)-\mu^2=\frac{\sigma_\varepsilon^2}{1-\varphi^2},$

$\textrm{var}(X_t) = \varphi^2\textrm{var}(X_{t-1}) + \sigma_\varepsilon^2,$

The autocovariance is given by : 

$B_n=\operatorname{E}(X_{t+n}X_t)-\mu^2=\frac{\sigma_\varepsilon^2}{1-\varphi^2}\,\,\varphi^{|n|}.$

二、移動平均模型 (MA, Moving Average Ｍodels)

序列可以由同期與過去的隨機項給予不同的權重來解釋，這稱為移動平均模型。移動平均模型在任何條件下都平穩。

1. 移動平均模型定義

The notation MA(q) refers to the moving average model of order q:

$X_t = \mu + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \cdots + \theta_q \varepsilon_{t-q} \,$

where μ is the mean of the series, the θ1, ..., θq are the parameters of the model and the εt, εt−1,..., εt−q are white noise error terms. 

三、ARMA模型 (AR, MA 兩者的混合)

ARMA模型由自迴歸模型（簡稱AR模型）與滑動平均模型（簡稱MA模型）為基礎「混合」構成。

1. ARMA模型定義

ARMA(p,q)模型中包含了p個自回歸項和q個移動平均項，ARMA(p,q)模型可以表示為：

$X_t = c + \varepsilon_t +  \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{j=0}^q \theta_j \varepsilon_{t-j} \ $

[用心去感覺] 金融領域的AR與MA

以金融領域來說，AR是過去的報酬，MA是過去的預測誤差；在效率市場下，我們預期 AR 或 MA 的參數是0。

統計上常用下面兩種方式做檢定：

• Ljung-Box Q 統計量：運用自我相關係數來檢定是否具有落後 p 期內的自我相關
• Breush-Godfrey LM 檢定(Serial Correlation LM Test；序列相關 LM 檢定)：運用殘差來檢定是否具有落後 p 期內的自我相關

四、ARIMA 模型 (Autoregressive Integrated Moving Average model)

非恆定的序列可以先經過差分的處理方式將其轉變為恆定序列，再透過ARMA的模型來預測。例如隨機漫步過程並沒有固定的平均水準，經過一階差分（first difference）後成為隨機震動所組成的數列，即變成平穩。ARIMA模型處理序列的一階動差，GARCH模型則處理二階動差。

ARIMA (p，d，q) 中，

• AR是"自回歸"，p為自回歸項數
• MA為"滑動平均"，q為滑動平均項數
• d為使之成為平穩序列所做的差分次數 (階數)，「差分」一詞雖未出現在ARIMA的英文名稱中，卻是關鍵步驟。

1. ARIMA模型定義

ARIMA (p，d，q) 模型是ARMA(p，q) 模型的擴展。ARIMA(p，d，q)模型可以表示為：

$\left(1 - \sum_{i=1}^p \phi_i L^i\right) (1-L)^d X_t = \left(1 + \sum_{i=1}^q \theta_i L^i\right) \varepsilon_t\,$

其中L 是滯後算子（Lag operator），$d \in \mathbb{Z}, d>0$

References

wiki : MA models

https://en.wikipedia.org/wiki/Moving-average_model

wiki : ARMA models
https://zh.wikipedia.org/wiki/ARMA%E6%A8%A1%E5%9E%8B

wiki : ARIMA models
https://zh.wikipedia.org/wiki/ARIMA%E6%A8%A1%E5%9E%8B

ARIMA模型

http://140.123.5.6/deptfin/course_data/Data/ch11_2.pdf

財務筆記: 常用的Econometrics II (part 39)

http://tomatokafka.pixnet.net/blog/post/56940242-%E8%B2%A1%E5%8B%99%E7%AD%86%E8%A8%98%3A-%E5%B8%B8%E7%94%A8%E7%9A%84econometrics-ii-(part-39)