Stationarity in Time Series¶
1. Fundamental Notation¶
To analyze the behavior of a time series, we define its moments (mean, variance, and covariance) as functions of time:
- Mean Function: \(\mu_{t} = E[Y_{t}]\). This represents the expected value of the variable at a specific time point \(t\).
- Variance Function: \(\sigma_{t}^{2} = \gamma_{t,t}\). We typically assume that the variance is finite (\(\sigma_{t}^{2} < \infty\)).
- Auto-covariance Function: \(\gamma_{t,s} = Cov(Y_{t}, Y_{s})\). This measures the linear dependence between values at two different time points, \(t\) and \(s\).
> Note: The term "Auto" signifies that the covariance is calculated between elements of the same series, unlike standard statistics which usually looks at two different variables (e.g., \(X\) and \(Y\)). - Auto-correlation Function (ACF): \(\rho_{t,s} = Corr(Y_{t}, Y_{s}) = \dfrac{\gamma_{t,s}}{\sigma_{t}\sigma_{s}}\). This normalizes the auto-covariance to a range between -1 and 1.
2. The Concept of Stationarity¶
In time series analysis, we deal with joint probability distributions. For a sequence \(X_{1}, X_{2}, \dots, X_{n}\):
* \(F_{X_{1}}(x_{1})\) represents the marginal Cumulative Distribution Function (CDF).
* \(f_{X_{1}}(x_{1})\) represents the marginal Probability Density Function (PDF).
* Joint PDF: Because time series observations are not independent, the joint PDF \(f_{Y_{t}, Y_{s}}(y_{t}, y_{s})\) is generally not equal to the product of their marginals (\(f_{Y_{t}}(y_{t}) \times f_{Y_{s}}(y_{s})\)).
The Problem: Handling joint PDFs is difficult because the distribution (or its moments) can change at every time point \(t\), making it nearly impossible to estimate parameters from a single realization of the data.
The Solution: We introduce Stationarity, the most common assumption in time series analysis. It assumes the process is in a state of "statistical equilibrium," meaning the underlying probability laws governing the process do not change over time.
3. Strict (Strong) Stationarity¶
A stochastic process \(Y(w, t)\) is considered strictly stationary if its joint distribution remains invariant under a shift in time.
- First-order Stationarity in Distribution: The marginal distribution stays the same regardless of the time shift \(k\):
$\(F_{Y_{t_{1}}}(y_{1}) = F_{Y_{t_{1}+k}}(y_{1})\)$ - Second-order Stationarity in Distribution: The joint distribution of any two observations depends only on the lag between them, not the absolute time:
$\(F_{Y_{t_{1}}, Y_{t_{2}}}(y_{1}, y_{2}) = F_{Y_{t_{1}+k}, Y_{t_{2}+k}}(y_{1}, y_{2})\)$ - \(n\)-th Order (Strict) Stationarity: For any set of time points \(\{t_{1}, t_{2}, \dots, t_{n}\}\), the joint CDF remains identical when the entire set is shifted by any value \(k\):
$\(F_{Y_{t_{1}}, \dots, Y_{t_{n}}}(y_{1}, \dots, y_{n}) = F_{Y_{t_{1}+k}, \dots, Y_{t_{n}+k}}(y_{1}, \dots, y_{n})\)$
In essence, Strong Stationarity implies that the entire joint distribution remains unchanged even if the time origin (the timestamp) is shifted.