Skip to content

Spectral Density Estimation

Estimation of the Spectral Density Function (SDF) allows us to move from theoretical models to practical data analysis, identifying the "power" behind various cycles in a time series.

Practical Applications

  • Signal Processing:
    • Noise Filtering: We can remove noise by filtering out high-frequency components identified via the SDF.
    • Restoration: Used to clean audio recordings or improve image quality.
  • Medicine and Biology:
    • ECG/EEG: Analyzing the spectral components of heart or brain waves for diagnosing conditions like arrhythmia.

Parametric Estimation

Parametric estimation assumes the time series follows a specific, known model (like an ARMA process). We estimate the parameters of that model to compute the SDF analytically.

Steps

  1. Model Selection: Choose an \(AR(p)\), \(MA(q)\), or \(ARMA(p,q)\) structure.
  2. Parameter Estimation: Estimate the coefficients (\(\phi, \theta\)) and noise variance \(\sigma^2\) using methods like Maximum Likelihood Estimation (MLE).

  3. Compute the SDF: Plug the estimated parameters into the theoretical formula for that model's spectrum.

  4. Pros:

    • Provides very smooth estimates.
    • Highly accurate for well-modeled stationary processes.
  5. Cons:
    • Strictly requires correct model specification; if the model is wrong, the spectrum is wrong.
    • May perform poorly for complex processes that don't fit standard ARMA forms.

Non-parametric Estimation

Non-parametric estimation does not assume a specific model structure. Instead, it attempts to directly estimate the SDF from the data itself.

Important

We try to estimate the SDF using a periodogram.

  • The Periodogram:
    • A fundamental non-parametric tool for SDF estimation.
    • It is a graph that provides a measure of the power (variance) of a signal at different frequencies.

The Periodogram

The periodogram \(I(\omega)\) is the sample analog of the spectral density.

\[ I(\omega) = \dfrac{1}{T}\left\lvert \sum_{t=1}^{T} y_{t}e^{-i\omega t} \right\rvert^{2} \]

Alternative Algebraic Form:
$$
I(\omega) = \dfrac{1}{T}[\text{Re}^2(\omega) + \text{Im}^2(\omega)]
$$
- where:
- \(\text{Re}(\omega) = \sum y_{t} \cos(\omega t)\)
- \(\text{Im}(\omega) = \sum y_{t} \sin(\omega t)\)

Note: Both the SDF and the periodogram are functions of the frequency \(\omega\). In this context, \(I(\omega)\) acts as an estimate of \(S(\omega)\).

Properties

  • Frequency Range:
    • Evaluated at discrete Fourier Frequencies: \(\omega_{k} = \dfrac{2\pi k}{T}\) for \(k = 0, 1, \dots, T-1\).
    • For real-valued series, the periodogram is symmetric; we usually only plot \(\omega \in [0, \pi]\).
  • Units:
    • Measured in variance per unit frequency (e.g., \(volts^{2}/Hz\)).
  • Bias and Variance:
    • Asymptotically Unbiased: \(E[I(\omega)] \approx S(\omega)\) as \(T\) gets large.
    • Inconsistency: The variance of \(I(\omega)\) does not decrease as \(T\) increases. This means the plot remains "jagged" even with more data.

Steps to Compute the Periodogram

  1. Fourier Transformation: Transform the data into the frequency domain using the Discrete Fourier Transform (DFT):
    $\(Y(\omega_{k}) = \sum_{t=1}^{T} y_{t} e^{-i \omega_{k}t}\)$
  2. Compute Power Spectrum: The periodogram is the squared magnitude of the DFT results, scaled by the sample size:
    $\(I(\omega_{k}) = \dfrac{1}{T} |Y(\omega_{k})|^2\)$
  3. Frequency Resolution: A longer series \(T\) provides better frequency resolution (ability to distinguish between two close frequencies):
    $\(\Delta f = \dfrac{1}{T \Delta t}\)$

Limitations of the Periodogram

  • Noisy Estimates: Because the variance doesn't decrease with sample size \(T\), the periodogram is an inconsistent estimator. It looks like a "hairy" or erratic plot.
  • Spectral Leakage: This occurs when the time series contains frequencies that do not align perfectly with the discrete Fourier frequencies. The power "leaks" out from the true frequency into adjacent frequency bins.
  • Resolution vs. Variance Tradeoff:
    • Higher frequency resolution (large \(T\)) \(\iff\) Increased variance/noise in the estimate.
    • To fix this, researchers often use Smoothing Techniques (like Daniel windows or WOSA) to reduce variance at the cost of some resolution.