Skip to content

L56 Machine Learning in TS

  • Why use it in TS?
    • flexibility
    • ability to capture complex patterns
  • Choose between models, each with its own strengths and weakness

ML relates to black-box models. So, we don't know what exactly goes into it.

Classical Machine Learning Models

  • Linear Regression
  • Support Vector Machines (SVMs)
    • TS classification and regression tasks
    • non-linear relationships
  • Random Forests and Gradient Boosting

Neural Networks

Deep learning model

  1. Recurrent Neural Networks (RNNs)
    • LSTMs (Long Short-Term Memory)
      • Long-term dependencies in sequential data
    • GRUs (Gated Recurrent Units)
      • simpler alternative with comparable performance
  2. Convolutional Neural networks (CNNs)
    • Useful for extracting local patterns (small batches) from TS, in combination with RNNs
    • Transformers
      • Attention Mechanism
      • Time Series Transformers (TST) (adaptations of transformer architecture for TS forecasting & classification)
    • Autoencoders: anomaly detection and feature extraction in TS
  3. Hybrid Models
    • Combine a traditional statistical models/machine learning techniques with Neural networks
    • ARIMA + Neural networks
      • ARIMA for mean relationship
      • NN to capture residual patterns
    • CNN-RNN Hybrids
      • Use CNNs to extract the features from input
      • RNNs for temporal (TS) modelling
    • DeepAR
      • Probabilistic forecasting model developed by Amazon using RNNs

Probabilistic & Bayesian Models

Provide uncertainty estimates in forecasts

  • Gaussian Process
    • non-parametric models for smaller datasets
  • Bayesian Structural Time Series
    • Useful for modelling trends, seasonality and causal inference

Specialized Models for TS

  • Facebook Prophet
  • N-BEATS
  • TFT → combine attention mechanism with interpretable forecasting

Ensemble Learning

  • Bagging and boosting: aggregating prediction from multiple models
  • Stacking: layering models. predictions of one serve as input to another

Classical ML models rely on structured feature engineering to transform sequential nature of TS into tabular format, suitable for algorithms.