Spectral Representation of a series¶

Spectral Representation Theorem (SRT)¶

Theorem

Any stationary time series can be expressed as a combination of sinusoidal functions of different frequencies, each with its own amplitude and phase.

\[ y_{t} = \int_{-\infty}^{\infty} e^{ i \omega t } Z(d\omega) \]

$y_{t}$ → Stationary time series.
$\omega$ → Angular frequency.
$Z(d\omega)$ → A complex-valued stochastic process (with orthogonal increments) which determines the contribution of frequency $\omega$ to $y_{t}$.

Spectral Density Function (SDF)¶

Definition of SDF

$S(\omega)$ describes how the variance of the time series is distributed across different frequencies.

SDF provides insight into the periodic components of the data.
It is widely used in signal processing, finance, and various other fields.

For a weakly stationary time series $y_{t}$, the SDF is the Fourier transform of the Autocovariance Function (ACF) $\gamma(h)$:

\[ S(\omega) = \dfrac{1}{2\pi}\sum_{h=-\infty}^{\infty} \gamma(h) e^{ -i \omega h } \]

where $\gamma(h)= E(y_{t}y_{t+h})$.

Properties of SDF¶

Symmetry: For a real-valued TS, $S(\omega)$ is symmetric: $S(\omega) = S(-\omega)$.
Non-negativity: $S(\omega) \geq 0$ for all $\omega$.
Total Variance: The integral of the SDF over the fundamental frequency range gives the total variance of the TS:
$$Var(y_{t}) = \gamma(0) = \int_{-\pi}^{\pi} S(\omega) d\omega$$
Periodicity: $S(\omega)$ repeats itself every $2\pi$ length.
Inverse Relationship: An inverse relationship exists between the ACF $\gamma(h)$ and the SDF $S(\omega)$:
$$\gamma(h) = \int_{-\pi}^{\pi} S(\omega) e^{ i \omega h } d\omega$$

Interpretation¶

Low frequencies ($\omega \approx 0$): Correspond to long-term trends or slow-moving components in the TS.
High frequencies ($\omega \approx \pi$): Correspond to noise or rapid fluctuations.
Peaks in $S(\omega)$: Indicate dominant periodic components at specific frequencies.¹

Examples¶

White Noise¶

$y_{t} \sim WN(0, \sigma^{2})$
ACF: $\gamma(h) = \begin{cases} \sigma^2 & h=0 \\ 0 & \text{otherwise} \end{cases}$
SDF: $S(\omega)= \dfrac{\sigma^{2}}{2\pi}$
Observation: The SDF is constant for all $\omega$. This implies that white noise has equal power across all frequencies (analogous to "white light" containing all visible colors).

AR(1) Process¶

$y_{t} = \phi y_{t-1} + \epsilon_{t}$, where $|\phi| \lt 1$ and $e_{t} \sim WN(0, \sigma^{2})$.
ACF: $\gamma(h) = \dfrac{\sigma^{2}}{1-\phi^{2}}\phi^{|h|}$
SDF: $S(\omega) = \dfrac{\sigma^{2}}{2\pi} \dfrac{1}{ | 1 - \phi e^{ -i\omega }|^{2}}$
Observation: If $\phi$ is close to 1 (high persistence), the low frequencies dominate, resulting in a "slow-moving" behavior.

MA(q) Process¶

The spectral power is limited to frequencies below a certain level, determined by $q$.
These are generally smoother than an AR process of a similar order because the series is based on a finite window of past shocks.

Random Walk (Non-stationary)¶

SDF: $S(\omega) = \dfrac{\sigma^{2}}{2\pi} \dfrac{1}{ | 1 - e^{ -i\omega }|^{2}}$ (Note: $\phi = 1$).
Observation: The SDF diverges ($S(\omega) \to \infty$) as $\omega \to 0$. This indicates a non-stationary process heavily dominated by low-frequency (infinite-horizon) components.

Question to consider: What would happen to the shape of $S(\omega)$ as the lag $h$ increases in the summation, and how does it affect the resolution of the peaks?

We can carve out the path of an SDF. There will be peaks in specific points in the SDF. Those peaks represent the dominant periodic components. ↩