Skip to content

Nonlinear TS Models

Introduction

Nonlinear Time Series (TS) models are extensions of traditional linear models designed to account for systems where relationships cannot be adequately captured by simple additive models. These are essential for modeling real-world data that exhibit structural shifts or complex behaviors.

Characteristics

Motivation: Why should we change from linear to nonlinear processes?

  • Nonlinearity: Relationships are not additive; they depend on interactions, powers, or specific thresholds.
  • Complex Dynamics: Ability to model chaos, bifurcations, and non-standard periodic behavior.
  • State Dependence: The influence of a variable on future values differs based on the current "state" of the system.
  • Non-stationarity: Statistical properties (mean and variance) may change over time in ways that a simple trend cannot explain.

Examples of Nonlinear Processes2

  • Threshold Models: TAR models, where dynamics change above or below certain limits.
  • Volatility Models: GARCH models, where variance is a nonlinear function of past shocks.
  • Nonlinear Dynamical Systems: Deterministic chaos1 (e.g., the Lorenz attractor).
  • Neural Networks: Recurrent Neural Networks (RNNs), LSTMs, and Transformer-based models.
  • Polynomial and Rational Models: Models where terms are squared, cubed, or divided.

Threshold Models

Threshold models are a class of nonlinear TS models where the dynamics of the system change based on whether the process crosses certain threshold limits. After a certain point (the threshold), the governing equations change. This is incredibly useful for capturing abrupt changes and regime shifts.

Key Concepts

  • Regimes: Distinct phases or states of a system.
    • The TS operates under different "regimes," which are subsets of data separated by thresholds.
    • In Threshold models, regimes are defined based on a variable crossing a specific threshold (e.g., \(Y_{t-d}\)).
  • Threshold Variable: The variable whose value determines the regime.
    • This is often a lagged value of the TS itself. For example, if we are modeling \(Y_{t}\), \(Y_{t-1}\) or \(Y_{t-2}\) might be the threshold variable.
  • Nonlinearity: These models exhibit piecewise linear behavior, which makes them nonlinear overall but still highly interpretable within each regime.

Regime Characteristics

  • Distinct Dynamics: Each regime has its own set of parameters and equations. For example, economic growth rates follow one pattern during an expansion and a completely different one during a recession.3
  • Transition Mechanism: Regimes change via the crossing of a threshold or a probabilistic switching mechanism.
  • Temporal Persistence: The system tends to persist in a regime for a period before transitioning, leading to a clustering of similar states.
  • System Nonlinearity: While linear within a regime, the abrupt or smooth transitions between regimes make the overall system nonlinear.

Types of Regime Transitions

The way a system moves from one regime to another defines the specific model type:

  1. Abrupt Transition (Discrete Switching):

    • Changes are instantaneous the moment the threshold condition is met.
    • Examples: TAR and SETAR models.
    • Uses: Economic recessions and recoveries, sudden market crashes, and booms.

  2. Smooth Transitions:

    • Changes occur gradually over a range of values of the threshold variable.
    • Examples: STAR models (using Logistic or Exponential transition functions).
    • Use cases: Gradual policy shifts or slow transitions between weather patterns.

  3. Probabilistic Transitions:

    • Transitions are governed by probabilities, often modeled as latent (hidden) variables rather than a fixed threshold.
    • Example: MSAR (Markov Switching) models.
    • Use cases: Stock market volatility clustering or hidden states in biological systems.

  1. Complex dynamics, abruptions in the models. ↩

  2. Not Nonlinear "Time Series" models specifically, just nonlinear models in general. ↩

  3. Expansion and recession are regimes. ↩