Time Series Analysis, Forecasting, and Machine Learning

Access the course code, notebooks, and data via the resources tab, using Code link and GitHub link; notebooks aren’t hosted on GitHub, and resources and data processing scripts are available.

Learn how to succeed in this course by asking questions in the Q&A, meeting prerequisites, and getting hands dirty with both conceptual and coding lectures.

handle temporary 403 errors by downloading the file in a browser and uploading it via Colab's file explorer, noting that public IPs may be blocked by the host.

Explore time series basics and a finance primer, defining time series, distinguishing modeling from forecasting, and examining shapes, tasks, transformations, metrics, and practical stock price simulations.

A time series is real-valued observations collected at regular, discrete time intervals, including vector time series; the lecture contrasts it with other sequence data using stock prices and brain signals.

Learn the difference between modeling and predicting in time series analysis, and how a functional form reveals why a series is mean reverting or unbounded, guiding forecasting and understanding.

Explore how data shapes affect library behavior and visualization in time series, from one-dimensional series to two-dimensional frames and three-dimensional n by t by d arrays.

Explore time series tasks, from one-step forecasts to multi-step and multi-output forecasts, covering forecast horizon, incremental vs multi-output methods, and regression vs classification.

Apply power, log, and Box-Cox transformations to time series data to stabilize variance, linearize trends, and improve forecasting; understand lambda choice and limitations with zero or negative values.

Compare power, log, and box cox transformations on airline passengers data, visualizing effects with plots and histograms and identifying the optimal lambda.

Learn common error metrics for time series forecasting, including mean squared error, root mean squared error, mean absolute error, r-squared, and scale-invariant measures like MAPE and SMAPE.

Explore transforming and analyzing financial time series, focusing on stock returns and log returns, and understanding open, high, low, close, adjusted close, volume data, dividends, and splits.

Simulate stock prices using log returns drawn from a normal distribution and a drift term, via Montecarlo simulations, to explore time-series behavior and the link to Black-Scholes and ARIMA modeling.

Examine the random walk and its hypothesis in finance, connecting price simulations to Gaussian noise, log prices, log returns, and ARIMA implications for forecasting.

Evaluate the naive forecast and random walk as baselines in time series forecasting. Understand training vs test performance to gauge overfitting and the limits of complex models.

Implement a naive forecast by predicting the previous close from S&P 500 data and evaluate it with SSE, MSE, RMSE, MAE, R-squared, and MAPE.

Define time series basics, contrast modeling and prediction, and explore data shapes in Python. Identify one-step, multi-step, and multi-output forecasts, data transformations, metrics, and practical applications.

Share your feedback through the suggestion box to improve this time series analysis, forecasting, and machine learning course, including background, difficulty, and requested topics like algorithms, CNNs, and transformers.

Explore exponential smoothing methods for forecasting time series, including simple exponential smoothing, the whole model, and the whole Winters' model, handling non trending, trending, and seasonal data.

gain intuition for exponential smoothing, from simple and exponentially weighted moving averages to Holt's linear trend and Holt-Winters seasonal models, and understand additive versus multiplicative forms.

Explore the simple moving average, a rolling mean over a fixed window that reveals recent time-series trends and enables mean and variance estimation.

Explore the simple moving average in code with a Colab notebook, compute rolling means for window sizes 10 and 50 on Google and Apple, and examine rolling covariance and correlation.

Learn the exponentially weighted moving average (exponential smoothing, or low pass filter) and its on-the-fly calculation, including alpha's role and how recent data dominate the weighted mean.

Explore the exponentially weighted moving average on the monthly airline passengers data, configuring alpha to 0.2, using IWM with adjust false, and validating with a manual check.

Bridge the gap between exponential smoothing and the winters' model, reframing smoothing as a forecasting method and exploring level, alpha, and univariate time series predictions.

Implement simple exponential smoothing as a forecasting model with statsmodels, covering data prep, initialization, fitting, predicting, alpha setting, and a train and test forecast.

Explore Holt's linear trend model, which adds a trend component to exponential smoothing to forecast lines. It uses level and trend equations to predict future values.

Demonstrates Holt's linear trend model in code using a Colab notebook: import the stats models class, fit the Holt object, and forecast train and test data with fitted values.

Explore the Holt-Winters theory, adding seasonal components to a level and linear trend using additive or multiplicative methods, and learn how to forecast with exponential smoothing in monthly time series.

Apply Holt-Winters exponential smoothing in a prepared Colab notebook with a train split and 12 monthly seasonal periods; compare additive and multiplicative setups using RMSE and MAE.

Explore walk-forward validation for time series, contrasting it with traditional train-test split and cross-validation, preventing overfitting and reflecting real-world forecasting.

Learn how to implement walk-forward validation in time series code, explore multiple option combinations, and identify the best configuration for airline passenger data using mean squared error.

Apply the Winters' model to champagne sales data, using a 12-month seasonal period, and compare train and test R-squared to evaluate forecasting performance.

Explore applying Winsor's model to stock prices across multiple tickers, perform a log transform of close prices, handle uneven trading days, and compare Holt's linear trend with naive forecasts.

Explore how the seven-day rolling simple moving average is used to smooth covid-19 counts, while data entry errors and backlogs create lag and distort public reporting.

Use a fast and slow simple moving average crossover to generate buy and sell signals in stock prices, noting signal lag and the need to optimize window sizes.

Explore exponential smoothing methods from simple moving average to the Winters' model, with additive or multiplicative seasonality and trend. Use walk-forward validation for time series forecasts.

Explore how linear state-space models model electrical, mechanical, economic, and biological systems in continuous and discrete time, enabling forecasting, control, and state estimation with the Cowman filter.

Learn the basics of arima, how to select model orders using plots, compare auto arima with classical methods, and apply to sales and stock prices with seasonal and exotic data.

Explore autoregressive AR(p) models, where past time series values feed predictions, contrasting AR with MA and I components in ARIMA, and transform data into a predictor matrix.

Explore the moving average model MA(q) within ARIMA, a linear function of past error terms rather than input data, and learn how to simulate MA processes.

Develop the Arima model by combining auto regressive, moving average, and integrated components, emphasizing differencing to achieve stationarity and illustrating the random walk special case.

Explore ARIMA in code using airline passenger data, differencing, log transformation, and model comparison to identify ARIMA(12,1,0) with logging as the best.

Assess time series stationarity with practical code and the Augmented Dickey-Fuller test, interpret p-values, and apply differencing to prepare ARIMA models.

Apply the augmented Dicky Fuller test to airline passengers data and stock series to distinguish stationary from non-stationary signals, using log transforms and first differences, and interpret p-values.

Explore the auto correlation function (acf) to determine the moving average order in arima by examining the acf plot, confidence intervals, and lag-based nonzero autocorrelations.

Use the partial autocorrelation function to determine the autoregressive order p in ARIMA models by identifying the highest significant non-zero lag in the pacf plot.

Explore acf and pacf plots in code to diagnose autoregressive models. The Colab notebook guides AR(1), AR(2), and AR(5) simulations and compares them with Gaussian noise.

Continue examining ACF plots for MA processes, generate MA1 to MA6 series, and interpret autocorrelation to guide ARIMA order selection in code.

compare manual arima rules with auto arima to let a computer automatically select the best time series model, including seasonal scREAMO with exogenous variables via PMed Yarema.

Auto arima selects the best time series model using aic and bic penalties with a stepwise search to balance complexity and accuracy.

this lecture applies auto arima to the airline passengers data, comparing logged and non-logged series, testing seasonal and nonseasonal orders, and evaluating models with AIC, forecast plots, and out-of-sample error.

Apply auto arima to stock prices, compare ARIMA models with naive forecasts, and evaluate train and test splits and 30-day forecast horizons with confidence bounds across multiple stocks.

Apply acf and pacf to returns with log returns to achieve stationarity. Across Google, Apple, IBM, and Starbucks, acf and pacf indicate arima(0,1,0), a random walk, highlighting model parsimony.

Apply auto arima to champagne sales data, comparing seasonal and non-seasonal models with a log-transformed series. The best forecast uses arima(12,1,1) after grid and stepwise search with walk-forward validation.

Forecast with ARIMA by structuring time series data correctly and avoiding test-data leakage. Recognize why copying the last value can mislead forecasts and how to evaluate out-of-sample accuracy.

Forecast beyond the test period without true data by treating train and test splits as out-of-sample emulation; the model does not use test data for forecasting, only to compute metrics.

Analyze how arima models enable modeling and forecasting of time series while revealing data structure. Apply auto arima with aic and bic to real data like stock prices and sales.

Explore vector autoregression, the multivariate counterpart to ARIMA, and how interdependent time series—like website visitors and web servers—affect forecasts, with theory and code, and model conversions.

Analyze vector autoregression and vector moving average theory, including VAR and VARMA, their matrix parameters, and identifiability concerns. Explore practical Python usage with statsmodels for model selection and forecasting.

Apply VARMA code with statsmodels in Python by loading a two-city temperature dataset, reformatting into a time-indexed multivariate series, handling missing values via interpolation, and visualizing results.

Continue building a VARMA model for temperature forecasting by preparing and scaling data, performing train-test splits, fitting the model, generating forecasts, and comparing R-squared against a baseline.

Implement VARMA in Python, test orders via information criteria, and forecast using prior time series values; compare with ARIMA on Auckland and Stockholm using squared errors and R-squared.

Learn how to apply varma in python to an econometrics dataset, including data import, time index setup, GDP growth via log difference, and the term spread analysis with forecasting.

Explore varma econometrics code by selecting gdp growth and term spread, preparing data, splitting train and test, fitting models, and comparing varma with arima and var for predictive value.

Use Granger causality to test whether past values of one time series improve forecast of another in a VAR model in Python; it signals prediction, not true causality.

Explore Granger causality testing on GDP growth and term spread to evaluate forecasting direction across lags and clarify that it measures forecasting ability, not true causality.

Explore how to convert a model from one form to another in ARIMA theory, including infinite sums and the equivalence of AR, MA, ARMA, and vector autoregressions.

Explore vector autoregressive and moving average models for multivariate time series, including VARMA and ARIMA approaches, with attention to training time, overfitting, and testing against baselines.

Explore applying machine learning to time series by transforming tasks into supervised learning problems and autoregressive models, focusing on linear models, SVMs, and random forests, with intuition, geometry, and extrapolation.

Learn that supervised learning is a geometry problem, using regression to fit lines or curves and classification to separate categories with a decision boundary.

Convert data into time series form by using past values as inputs and future values as targets, enabling autoregressive and multi-step forecasts with nonlinear models.

Learn how linear regression fits a line of best fit for two-dimensional data, extends to multiple inputs with weights and intercept, and links to auto regressive time series.

Explore logistic regression as a classifier that turns linear model scores into probabilities with the sigmoid function, predicting Y=1 vs 0 and extending to multiclass via the argmax.

Explore the geometry of support vector machines for classification and regression, including maximum margin, support vectors, epsilon-insensitive loss, and the kernel trick with the rbf kernel.

Explore the random forest, an ensemble of hundreds of decision trees that improves tabular data predictions by voting for classification and averaging for regression, reducing overfitting.

Examine extrapolation in time series by showing how machine learning models fail to predict outside the training range on stock prices, highlighting stock returns as a potential alternative.

Explore turning a time series into supervised data, applying linear and other models for one- and multi-step forecasts on airline passengers data, and compare machine learning with differencing.

Make the airline passengers time series stationary by first difference, forecast delta values for one-step and multi-step forecasts, and reconstruct the series by summing deltas with the last observed value.

Explore time series forecasting in code by differencing log passengers and starting from index one. Compare linear regression, random forest, and a svr wrapper for one-step and multi-step forecasts.

Apply the existing code to the champagne sales time series, showing that only the dataset changes and the log transform is used. Explore one-step and multi-step forecasts with linear regression.

Apply the existing time series and machine learning workflow to stock prices and returns, testing linear and nonlinear forecasts. Discover that models yield near-zero r-squared, showing stock returns resist forecasting.

Predict IBM stock movement direction using a one-step, log-return based time series with 21 lags, comparing logistic regression, SVM, and random forest to assess overfitting.

Apply machine learning to time series by converting data for supervised methods, perform multi-output forecasts, and compare linear and non-linear autoregressive forecasters with benchmarks.

Explore deep learning for time series analysis by learning artificial neural networks, their activation functions, and TensorFlow-based implementation for multivariate data.

Explore how linear and logistic regression form the building blocks of neural networks, revealing how weights, biases, and the sigmoid activation model neuron-like computation.

Explore forward propagation in neural networks, where inputs pass through wide and deep layers with vectorized weights and sigmoid activation to yield binary predictions, and learn hierarchical feature transformations.

Explore why neural networks matter for non-linear decision boundaries, showing how multiple neurons learn nonlinear features automatically, outperforming manual feature engineering and single neurons.

Analyze activation functions in deep learning, from sigmoid and tanh to ReLU, exploring vanishing gradients, zero-centering, and variants like leaky ReLU and ELU.

Learn multiclass classification and how softmax converts final-layer activations into a probability distribution over k categories. Contrast with sigmoid for binary tasks and apply to image and handwriting recognition.

Build a neural network with TensorFlow using the Keras API, detailing model creation, compile, fit with epochs, history, and predictions, via the functional API.

Implement a feedforward ann for time series forecasting on airline passengers data, producing one-step, incremental multi-step, and multi-output forecasts using log transform, differencing, and supervised datasets.

Demonstrate a feedforward neural network to predict stock prices and returns by standardizing log returns, building supervised data, and evaluating one-step and multi-step forecasts with inverse scaling.

Explore the human activity recognition dataset, a multivariate time series from smartphone accelerometer and gyroscope, with six activities and 128 measurements per sample, and compare feature-based, time-series, and hybrid models.

Learn to handle multivariate time series for human activity recognition by building a multi-tailed neural network with separate inputs per dimension, merging features via concatenation, and training with multiple inputs.

Explore human activity recognition data from the UCI dataset through a step-by-step examination of train and test folders. Note features, labels, inertial signals, and the 7352-by-128 data shape for training.

Load the nine parallel time series, reshape for training and testing, and build a multi-input neural network with nine mini networks whose features concatenate before a final six-class output.

Compare static-feature based models for human activity recognition, standardize features, and evaluate logistic regression, SVM, and random forest, showing feature engineering can outperform time-series models.

Combine the multi input time series cnn and the feature based cnn via concatenation to deliver a single neural network for human activity recognition.

Explore how a model learns from linear regression to gradient descent, detailing mean squared error, gradients, learning rate, and the role of automatic differentiation in training.

Summarizes artificial neural networks, from basic feedforward and multi-input architectures for multivariate time series to nonlinear activations and optimization, applied to airline passengers, stock returns, and human activity recognition.

The lecture introduces convolutional neural networks (CNNs) and shows how to forecast and classify time series, using images for intuition and code-based practice.

Learn how convolution maps input images to output images using a filter or kernel, enabling blurring and edge detection. Explore padding and modes like valid, same, and full.

Explore convolution from a pattern-matching perspective, viewing it as a sliding dot-product filter that seeks patterns in data, relates to cosine similarity, cross correlation, and Pearson correlation.

Explore the equivalence of convolution and matrix multiplication, and learn how weight sharing creates fewer parameters for translational invariance in a pattern finder across images.

Extend convolution to color images by using three-dimensional inputs and filters, producing multi-channel feature maps that capture multiple features; stack outputs for deeper layers and apply bias and activation.

Explore how convolution applied to time series smooths data through Gaussian-like filters, relates to moving averages and exponential smoothing, and connects autoregressive models with CNNs, via autocorrelation and cross-correlation.

Explore the cnn architecture from convolutional and pooling layers to dense layers, and learn how hierarchical feature maps and global pooling enable efficient image and time series analysis.

Prepare cnn code for time series analysis by reviewing convolutional layers, activation, and pooling syntax; build, compile, train, and predict on time series data, including 1d and 2d cnn variants.

Implement cnn-based time series forecasting with TensorFlow to predict airline passenger counts using one-step, multi-step, and multi-output forecasts, detailing conv layers and pooling.

Learn CNN-based human activity recognition on time-series data and a parallel, combined model that integrates tabular features for improved accuracy.

Convolutional neural networks act as pattern matchers through convolution and cross-correlation, using convolution filters as templates on input data to unlock CNN-driven time series insights.

Explore recurrent neural networks and their backwards-in-time connections, from simple models to long-term memory units like LTM/LSM and Ajamu, with diagrams and hands-on code.

Explore how simple recurrent neural networks use previous hidden states and inputs in the Elman unit to influence current predictions through recurrence and matrix operations.

Explore how a simple rnn (Elman unit) handles many-to-one tasks like spam detection and sentiment analysis, and many-to-many tasks like parts of speech tagging, with dense layers and shared weights.

Explain the relation between linear state space models and nonlinear ANN equivalents, detailing how hidden states evolve with A and B and observations with C and D, learned by ML.

Prepare and preview RNN code for time series models by learning the syntax to build, compile, and predict with simple arnet layers and return sequences, with LSTM or GRU options.

Explore rnn implementation by a manual forward pass in Colab, tracking shapes from input dimension d through hidden units m to outputs k with weights w_x, w_h, w_o and tanh.

Examine GRU and LSTM concepts, explain the vanishing gradient problem, and show how update and reset gates create a convex, weighted mixture to remember or forget past states.

Discover how LSM units use forget, input, and output gates to manage the cell and hidden states for long-term dependencies. Compare LSM to GIU, guided by results that favor LSM.

We demonstrate time series forecasting with LSTMs, covering one-step, incremental multi-step, and multi-output forecasts, exploring variations like final hidden state, global max pooling, and stacked LSTMs.

Explore LSTMs for time series classification in code, comparing models that use only time series data with models that fuse time series and static features for human activity recognition.

Recurrent neural networks struggle for time series forecasting, while ARIMA and other classical methods often perform better, underscoring experimentation over theory using the M3 data.

Explore the theory and practical use of recurrent neural networks for time series, implemented in code with TensorFlow and JAX, and evaluated through experiments to find what works.