Cross-covariance & Cross-correlation¶

Understanding the relationship between multiple time series requires extending the univariate concepts of covariance and correlation into a matrix framework. This allows us to capture dependencies both within a single series and between different series at various lags.

1. Fundamentals: Random Vectors and Dispersion¶

A random vector is a collection of random variables. For a vector $X = (X_{1}, X_{2}, \dots, X_{p})'$, we define:
* Mean Vector: The vector of expected values for each component, $E(X) = (\mu_{1}, \mu_{2}, \dots, \mu_{p})'$.
* Variance-Covariance Matrix (Dispersion Matrix): Captures the variances of each variable and the covariances between all pairs:
$$\Sigma = E[(X-\mu)(X-\mu)'] = \begin{bmatrix} \sigma_{11} & \dots & \sigma_{1p} \\ \vdots & \ddots & \vdots \\ \sigma_{p1} & \dots & \sigma_{pp} \end{bmatrix}$$
* Cross-Covariance Matrix ($\Sigma_{xy}$): Measures the relationship between two different random vectors $X$ and $Y$:
$$\Sigma_{xy} = E[(X-\mu_{x})(Y-\mu_{y})']$$
The $ij$-th element represents $Cov(X_{i}, Y_{j})$.

2. Stationary Multivariate Time Series¶

A vector process $Y_{t} = (Y_{1t}, \dots, Y_{kt})'$ is strictly stationary if the joint probability distribution of any set of vectors is invariant to a time shift $l$. In a stationary process:
* The Mean Vector $\mu$ is constant over time.
* The Covariance Matrix depends only on the lag $l$, not on the absolute time $t$.

3. Cross-Covariance and Correlation Functions¶

For a multivariate process, we distinguish between relationships within a single series and between two distinct series.

Autocovariance and Cross-covariance¶

Autocovariance ($\gamma_{ii}(l)$): The covariance within the $i$-th series at lag $l$.
$$\gamma_{ii}(l) = E[(y_{it}-\mu_{i})(y_{i,t+l}- \mu_{i})]$$
Cross-covariance ($\gamma_{ij}(l)$): The covariance between the $i$-th series and the $j$-th series at lag $l$.
$$\gamma_{ij}(l) = E[(y_{it}-\mu_{i})(y_{j,t+l}-\mu_{j})]$$

Cross-correlation ($\rho_{ij}(l)$)¶

This normalizes the cross-covariance by the standard deviations of the two series:
$$\rho_{ij}(l) = \dfrac{\gamma_{ij}(l)}{\sqrt{\gamma_{ii}(0)\gamma_{jj}(0)}}$$

Matrix Representation¶

The Cross-Covariance Matrix at lag $l$, denoted $\Gamma(l)$, organizes all these relationships into a $k \times k$ matrix:
$$\Gamma(l) = E[(Y_{t}-\mu)(Y_{t+l}-\mu)']$$
The corresponding Correlation Matrix $\rho(l)$ can be calculated as:
$$\rho(l) = V^{-1/2} \Gamma(l) V^{-1/2}$$
where $V = diag(\gamma_{11}(0), \gamma_{22}(0), \dots, \gamma_{kk}(0))$ contains the variances of each series.

4. Vector White Noise Process¶

A sequence of random vectors $\{ e_{t} \}$ is a Vector White Noise process, denoted $e_{t} \sim WN(0, \Sigma)$, if:
* $E(e_{t}) = \vec{0}$
* The covariance matrix $\Gamma(k)$ satisfies:
$$\Gamma(k) = \begin{cases} \Sigma, & k=0 \\ 0, & k \neq 0 \end{cases}$$
This implies that while components within the same vector $e_{t}$ can be correlated (via $\Sigma$), there is no correlation between vectors at different time points.