ACF & PACF for Specific TS Processes

Mathematical derivations of the Autocorrelation Function (ACF) allow us to identify the underlying stochastic process by observing how correlations decay over time.

1. Autoregressive Process: AR(1)

For an \(AR(1)\) process, as explored in L8__Autocorrelation and the Partial Autocorrelation Functions, we can derive the auto-covariance recursively:
$\(\gamma_{k} = E(Y_t Y_{t-k}) = E[(\phi_{1}Y_{t-1} + e_{t})Y_{t-k}] = \phi_{1}E(Y_{t-1}Y_{t-k}) = \phi_{1}\gamma_{k-1}\)$

By continuing this recursion, we find:
$\(\gamma_{k} = \phi_{1}^k \gamma_{0}\)$

The ACF of AR(1):
$\(\rho_{k} = \dfrac{\gamma_{k}}{\gamma_{0}} = \phi_{1}^k\)$

Simulation Insight

In a simulated \(AR(1)\) process with a coefficient (\(\phi_{1}\)) = 0.6, the plot looks like a normal TS graph. The ACF is downward sloping because \(\phi_{1}^k\) keeps getting smaller with \(k\). ACF gradually decays

2. Moving Average Processes: MA(q)

Unlike the AR process, the MA process has a "finite memory," meaning the correlation cuts off after a certain number of lags.

• General Properties:
  • Mean: \(E(Y_{t}) = c = \mu\).
  • Variance: \(V(Y_{t}) = \sigma_{e}^{2} \sum_{j=0}^q \theta_{j}^{2}\) where \(\theta_{0}=1\).
• MA(1) Process:
  • \(\gamma_{1} = Cov(Y_{t},Y_{t-1}) = \theta_{1}\sigma_{e}^{2}\).
  • For all \(k > 1\), \(\gamma_{k} = 0\).
  • ACF:
    $\(\rho_{k} = \dfrac{\gamma_{k}}{\gamma_{0}} = \begin{cases} \dfrac{\theta_{1}}{(1+\theta_{1}^{2})} & \text{if } k = 1 \\ 0 & \text{if } k \gt 1 \end{cases}\)$
• MA(2) Process:
  • \(\gamma_{1} = Cov(Y_{t},Y_{t-1}) = \theta_{1}(1+\theta_{2})\sigma_{e}^{2}\).
  • \(\gamma_{2} = \theta_{2}\sigma_{e}^{2}\).
  • For \(k > 2\), \(\gamma_{k} = 0\).

Identification Rule

For any \(MA(q)\) process, \(\gamma_{k} = 0\) for all \(k > q\). Thus, the ACF function can help identify the possible models to fit into a certain component. ACF cuts off after \(q\)

3. Random Walk

A Random Walk is a classic example of a non-stationary process where the mean and variance depend on the time point \(t\):

• Mean: \(E(Y_{t}) = t\mu\).
• Variance: \(V(Y_{t}) = \gamma_{0}= t\sigma_{e}^{2}\).
• Auto-covariance: \(\gamma_{k}= (t-k)\sigma_{e}^{2}\).
• ACF: \(\rho_{k} = \sqrt{ 1 - \dfrac{k}{t} }\).

---