L43 Spectral Density Estimation

Practical Applications

• Signal processing
  • Noise filtering → remove noise by filtering out high-frequency components using the SDF
  • cleaning audio recordings/ improve image quality
• Medicine and biology
  • ECG: diagnosing arrhythmia

Parametric Estimation

• Specific model for the TS
  • ARMA or ARIMA
  • estimate parameters of that model to compute SDF

Steps

1. Model selection: \(AR(p)\) or \(MA(q)\) or \(ARMA(p,q)\)
2. Parameter estimation: estimate \((\phi,\theta)\) using methods like MLE etc

3. Compute the SDF using the analytical formula

4. Pros

   • Provides smooth estimates
   • Suitable for well-modeled stationary processes
5. Cons
   • Requires correct model specification
   • May not work well for complex or unknown processes

Non-parametric Estimation

• Doesn't assume a specific model
• Directly estimate the SDF from the data

We try to estimate the SDF using a periodogram.

• Periodogram
  • A fundamental non-parametric method for estimation of SDF
  • It is a graph → provides measure of power (variance) of a signal at different frequencies

The Periodogram

\[ I(\omega) = \dfrac{1}{T}\left\lvert \sum_{i=1}^{T} y_{t}e^{ -\omega t } \right\rvert^{2} \]

Alternatively

\[ I(\omega) = \dfrac{1}{T}[\mathrm{Re}^{2}(\omega) + \mathrm{Im}^{2}(\omega)] \]

• where
  • \(\mathrm{Re}^{2}(\omega) = \sum y_{t} \cos \omega t\)
  • \(\mathrm{\mathrm{Im}}^{2}(\omega) = \sum y_{t} \sin \omega t\)

Both SDF and periodogram are function of the frequency, \(\omega\) and \(I(\omega)\) is an ESTIMATE of \(S(\omega)\)

Properties

• Frequency Range
  • Evaluated at discrete frequencies
  • \(\omega_{k} = \dfrac{2\pi k}{T}\) where \(k = 0,1,\dots,T-1\)
  • For real-valued TS, periodogram is symmetric so \(\omega \in [0,\pi]\)
• Units
  • Units are variance per unit frequency
  • e.g. \(\dfrac{volts^{2}}{Hz}\)
• Bias and Variance
  • Asymptotically unbiased: \(E[I(\omega)] \approx S(\omega)\)
  • Variance: not consistent, variance doesn't decrease with increasing \(T\)

Steps to compute the Periodic

1. Transform the frequency domain using DFT
   • \(Y(\omega_{k})=\sum_{t=1}^{T} Y_{t} e^{ -i \omega_{k}t }\)
2. Compute the power spectrum
   • Periodogram = squared magnitude of the DFT scaled by \(1/T\)
   • \(I(\omega_{k}) =\dfrac{1}{T}[Y(\omega_{k})]^{2}\)
3. Frequency resolution
   • longer \(T\) provides better frequency resolution
   • \(\triangle f = \dfrac{1}{T \triangle t}\)

Limitations of Periodogram

• Noisy estimates
  • \(I(\omega)\) ka variance doesn't decrease with sample size, \(T\)
• Spectral Leakage
  • When TS contains frequencies not aligned with discrete Fourier frequencies, power leaks into adjacent frequencies #what?
• Resolution vs Variance tradeoffs
  • High-frequency resolution \(\iff\) increased variance