Skip to content

L58 Other ML Models for TS

k-Nearest Neighbors (KNN)

  • adapted for TS forecasting or classification
  • primarily used to identify the most similar historical patterns (neighbors)
  • use their outcomes to predict future values

Steps

  1. Convert TS to a supervised problem
    • essentially label each datapoint
    • lagged features
  2. Similarity measure
    • Euclidean distance (similarity between TS segment is calculated)
    • Each time step (row) is treated as a vector in \(p\)-dimensional space
  3. Prediction
    • For a given test point, algorithm finds the \(k\) nearest neighbors from the training set
    • Output is the
      • average (for regression)
      • majority class (for classification)

Key Components

  1. Feature Engineering
    • Lagged features
    • Optionally, include rolling statistics (moving averages)
  2. Hyperparameters (strike a balance)
    • \(k\) → number of neighbors to consider
    • Distance metric → Euclidean distance / Manhattan distance
  3. Normalization
    • To ensure all features contribute equally to the distance calculation
  4. Model evaluation
    • Regression: MSE (Mean Squared Error), MAE (Mean Absolute Error)
    • Classification: Precision, Recall, Accuracy

Advantages

  • simplicity
  • non-parametric
  • handles non-linearity

Limitations

  • Computational cost: high for large datasets (for computing distances)
  • Curse of Dimensionality: (carefully select the number of lagged features, and normalize data)
  • No explicit model: (memory-based, doesn't create a model) \(\implies\) less efficient for frequent predictions

Example - Pattern Recognition

  • Recurring events, anomaly detection
  • This pattern can be spotted and identify which subgroup of the time series is similar to this pattern
  • e.g. Stock price movements, vibrations etc
  • Brief steps
    1. Define a reference pattern
    2. Segment the time series into (non) overlapping windows to capture local patterns
    3. Extract features
      • Compute features (mean, variance, slopes, Fourier coefficients) to represent sub-sequences
    4. Train KNN
      • "Pattern Found" vs "No Pattern"
    5. Test on unseen data

Other Applications of Pattern Recognition

  • Abnormal vibration or temperature fluctuations in machine sensor data
  • Head-and-shoulders or cup-and-handle patterns in stock prices
  • Specific patterns in ECG signals (arrhythmias)
  • Detect rainfall patterns in weather TS

Naive Bayes for TS

  • Classification problem based on Bayes theorem
    • lagged values are conditionally independent given the class label
  • To identify if a given time segment belongs to a "normal" or "anomalous" class