Financial Engineering and Artificial Intelligence in Python

Learn where to access course code, notebooks, and data through the resources tab, the code link, and the GitHub link, with guidance on two code formats and common retrieval mistakes.

Master three guidelines for this course: use the anonymous Q&A, meet prerequisites, and get your hands dirty—take handwritten notes for conceptual and code what you see in coding lectures.

Define financial data and explore time series analysis and machine learning with stock returns. Examine distributions, hypothesis testing, and volatility clustering in technical analysis as you simulate stock prices.

Learn to download stock data with yfinance, save it as CSV files in a data folder, and assemble a symbol-labeled data frame for S&P 500 stocks.

Load Kaggle S&P stock dataset with missing data, inspect with pandas, filter IBM, and build date-indexed close price matrix with each stock as a column, then save and reload.

Explore why data goes missing in financial time series and apply forward filling or backward filling to preserve integrity, avoiding linear interpolation that uses future information.

Develop and practice handling missing data in financial time series using pandas: drop rows with all missing values, forward and backward fill, count missing values, and normalize prices for comparison.

Explore how the adjusted close accounts for stock splits and dividends, compare it to the close price, and learn how splits impact returns calculations in practice.

Compare close and adjusted close prices using Yahoo Finance data, plotting Google and Apple to verify stock splits and dividends are reflected, and compare with the Kaggle data set.

Calculate stock returns in python with pandas and numpy: align close and previous close using shift, compute returns and log returns, and visualize distributions with histograms and basic statistics.

Use the cu-cu plot (quantile-quantile plot) to compare sample quantiles with theoretical normal quantiles, interpret straight lines as good fit, and recognize heavy tails or skew when patterns deviate.

Examine how the t distribution, with heavier tails than the normal, models returns, explore its degrees of freedom, location, and scale, and fit parameters via maximum likelihood.

Model returns with the t-distribution in Python by fitting degrees of freedom, location, and scale, then compare histograms, the t pdf, and Cucu plots for returns and log returns.

Explore skewness and keratosis as measures of distribution shape and tail heaviness; negative skew signals potential extreme negatives, while excess keratosis indicates heavier tails than normal, with stock returns.

Learn how confidence intervals quantify uncertainty in the expected return by linking the sample mean, variance, and sample size to 95% bounds via normal and t distributions.

Compute the mean and standard deviation of stock returns, then plot the confidence interval as mean plus or minus 1.96 times s divided by sqrt n on the histogram.

Explore the API approach to statistical testing, learn null and alternative hypotheses, p values, and when to reject the null using z and t tests with one- and two-sample examples.

Describe covariance and correlation in stock returns, compute the covariance and correlation matrices, and interpret variance and linear relationships for portfolio insights.

Analyze stock return covariance and correlation through code. Compute returns, and visualize with a seaborn pair plot to reveal relationships among Apple, Google, IBM, Netflix, and Starbucks.

Calculate alpha and beta in Python by comparing Apple stock to the spy, plotting returns and the line of best fit, and assessing risk via volatility.

Explore the Gaussian mixture model for density estimation and clustering, showing how a multimodal, heavier-tailed mixture of Gaussians helps model returns and learn parameters via maximum likelihood and EM.

Highlight volatility clustering in stock returns via time-series plots, illustrating that high volatility clusters with high and low with low, measured by standard deviation for Apple, Google, IBM, Starbucks, Netflix.

Explore price simulation to test algorithm robustness under uncertainty by replaying historical returns or sampling from a fitted distribution, and use the price evolution formula to generate future prices.

Simulate stock prices in code using two methods—sampling from existing returns and a fitted t distribution—starting from the final close price.

Model stock return as a non time varying random variable, compare the t distribution to the normal, study skewness, correlation, covariance, alpha and beta, volatility clustering, and price simulation.

Explore time series analysis using Holt-Winters exponential smoothing and ARIMA to model stock price evolution, balancing algorithm choices with finance domain knowledge.

Analyze the efficient market hypothesis and its weak, semi-strong, and strong forms; show that prices reflect information and beating the market is not consistently possible. Explore how time series analysis and modeling diverge from naive predictions in finance.

Explore the random walk hypothesis as a case of ARIMA in time series and compare it with the efficient market hypothesis; connect log prices to Gaussian noise and Markov chains.

Master the simple moving average as a rolling mean over a fixed time window, with a hands-on example and pandas rolling for calculating mean, variance, and other statistics.

Explore computing simple moving averages in Python, using pandas and numpy to visualize time series, log returns, rolling covariances, and evolving correlations between Google and Apple.

Learn the exponentially weighted moving average, also called exponential smoothing or a low pass filter. The online update uses alpha to weight new samples and decay past data exponentially.

Explore the exponentially weighted moving average code in a Colab notebook, using an alpha parameter to plot and validate the WEMA on airline passenger data.

Extend exponential smoothing with a linear trend to forecast future values. Holt's model combines a level, a trend, and a linearly increasing forecast with alpha and beta learned via optimization.

Apply Holt's linear trend model in code using a Colab notebook; fit the model, assign fitted values, split data into train and test, and forecast trends in the time series.

Explore the Holt-Winters time series model, adding seasonal components to trend and level, with additive and multiplicative methods, and learn its forecast and code implementation.

Apply Holt-Winters exponential smoothing with additive trend and seasonality, fit on train, forecast on test, compare multiplicative variants, and favor the additive model based on RMSE and MAE.

Explore autoregressive models and AR(p) within ARIMA frameworks, contrasting them with exponential smoothing, and learn how past values serve as predictors in time series.

Learn how to determine time series stationarity and apply the augmented dickey fuller test to decide ARIMA differencing (p, d, q) for robust forecasting.

apply the augmented Dickey-Fuller test to airline passenger data and stock time series to assess stationarity, using differencing, log transforms, and the ATF helper to interpret p-values.

Explore the auto correlation function (AKF) to determine the moving average order in ARIMA, using stationary assumptions, lag-based analysis, and confidence intervals to identify non-zero lags.

Use the partial autocorrelation function to choose the AR order p in ARIMA, interpreting significant non-zero lags as a guide to include AR terms up to p.

Examine acf and pacf in code using a Colab notebook to plot autocorrelations of autoregressive processes and interpret confidence bounds.

Continue exploring ACF in code by generating MA processes and plotting their autocorrelations to test ARIMA order selection, including MA(1), MA(2), MA(3), and MA(6) with confidence bounds.

Apply auto arima to airline passengers data, compare log-transformed and original series, and evaluate seasonal versus non-seasonal models using train-test split, AIC, BIC, and forecast plots.

Apply ARIMA and auto ARIMA to stock prices, train/test split forecasts, and compare with naive benchmarks to reveal when patterns predict or fail.

Using acf and pacf to select arima orders for stock returns, the lecture shows log returns are largely non-autocorrelated, yielding p=0 and q=0 and a random walk model.

Explore classical time series analysis from moving averages and exponentially weighted moving averages to Holt-Winters seasonality and ARIMA, including stationarity testing with the Augmented Dickey-Fuller test.

Explore modern portfolio theory and the capital asset pricing model, learning diversification, mean-variance optimization, and how linear and quadratic programming craft optimal portfolios under constraints.

Explore how the S&P 500 uses cap-weighted weights based on market capitalization and how the committee selects constituents by minimum market cap, trading volume, and NYSE or NASDAQ listing.

Explore risk in finance through the lens of standard deviation, linking volatility to potential errors in return predictions, and discuss portfolio optimization and modern portfolio theory for choosing assets.

Learn how diversification reduces risk by combining two independent stocks with the same return, lowering variance to half and decreasing standard deviation to the square root of two.

Describe a portfolio with weights summing to one, explain short selling via negative weights, and derive the portfolio mean and variance using the covariance matrix.

Visualize and simulate portfolio risk and return using a Monte Carlo approach with synthetic data, softmax weights, and covariance-based risk, preparing for real-stock analysis.

Visualize random portfolios with six-month S&P data in Colab using Monte Carlo simulation, compute returns and covariance, and examine the Markowitz bullet, showing diversification reduces risk and short selling constraints.

Minimize portfolio variance under a target return and weight constraints using mean-variance optimization, solved as a quadratic program with scipy minimize and SLSQP.

Explore the efficient frontier through mean-variance optimization, deriving minimum-variance portfolios for target returns on the risk-return plot. Learn alternative constrained, quadratic, and risk aversion formulations to balance return and variance.

Explore the global minimum variance (GMV) portfolio, at the tip of the Markowitz bullet, minimizing variance using covariance and ignoring expected return.

This lecture shows how to find and plot the GMV portfolio in code by minimizing portfolio variance, computing return, and visualizing the GMV on the efficient frontier.

Learn how the Sharpe ratio measures excess return per unit of risk to compare efficient portfolios, incorporating the risk-free rate, annualization, and practical code-based implementation.

Explore how the capital asset pricing model ties market risk to expected returns through the capital market line and security market line, highlighting beta, alpha, and mispricing.

Conclude how to build a diversified portfolio to mitigate risk. Explain expected return, variance, the efficient frontier, and optimization via linear and quadratic programming.

Introduce algorithmic trading with trend following using the simple moving average, then apply a machine learning model to predict tomorrow’s return and guide buy or sell decisions with reinforcement learning.

Discover a trend-following trading strategy using fast and slow simple moving averages to generate buy and sell signals from crossovers, demonstrated with paper trading and full investment decisions.

implement a trend following strategy in code using a Colab notebook, compute log returns, moving averages, and buy/sell signals, then compare performance to buy and hold with a Sharpe ratio.

This lecture introduces machine learning based trading strategies that predict tomorrow's stock return using a predictive model, updating positions daily to buy, sell, or hold.

Train a logistic regression classifier to predict tomorrow's return sign using binary targets and a regularization penalty, then evaluate train and test scores and compare with buy and hold.

Plug in a random forest classifier to machine learning based trading and compare test performance to illustrate how powerful models must be paired with robust signals for reliable results.

Learn algorithmic trading with trend following and moving averages, then compare linear, logistic, and random forest models, emphasizing data, robustness, and upcoming reinforcement learning.

Explore reinforcement learning and its difference from supervised learning, highlighting how time and planning toward a goal guide actions and the need for human-labeled data.

Define the core reinforcement learning concepts: environment, agent, episode, state, action, and reward, through tic tac toe, breakout, and maze examples.

Learn how to encode states and actions in code and define policies that maximize rewards. Explore discrete versus continuous states and probabilistic policies with epsilon-greedy.

Explore Markov decision processes by linking agents, environments, states, actions, and rewards within a probability framework. See how the Markov assumption and state transitions define environment dynamics for reinforcement learning.

Define the agent's objective as maximizing the sum of future rewards over an episode. Use gamma discounting to weight future rewards and define the return recursively in infinite-horizon mdps.

Explore reinforcement learning by contrasting prediction and control problems, define state and action value functions, and show how optimal policies and value functions are derived through evaluation and policy search.

Explore solving the Bellman equation with reinforcement learning through Monte Carlo sampling to estimate state values, compare prediction and control problems, and learn from episodic returns.

Explore applying Monte Carlo methods to reinforcement learning, evaluating and improving policies with Q(s,a) and V(s), via generalized policy iteration and exponentially decaying average updates.

Explore the explore-exploit dilemma in reinforcement learning with epsilon-greedy action selection, balancing sampling new actions to improve q-values with greedy exploitation of known best actions.

Learn how reinforcement learning solves MDPs with temporal difference methods, bootstrap estimates, and online updates, then apply off-policy Q-learning with epsilon-greedy policies to derive the optimal policy.

Learn reinforcement learning by implementing algorithms from basics, mastering dynamic programming, Monte Carlo, and temporal difference methods, then transitioning from linear models to deep learning for function approximation.

Reimplement trend-following as a reinforcement learning setup with an environment and agent, using fast and slow indicators and actions buy, sell, or do nothing, with returns when invested.

Revisit a trend-following strategy by implementing a reinforcement learning environment and agent, using fast and slow moving averages, log returns, and buy/sell/hold actions to train and test on split data.

Explore Q-learning in an algorithmic trading context, using a tabular Q-function to estimate future rewards, update via an exponentially weighted moving average, and balance exploration and exploitation with epsilon-greedy actions.

Represent states by discretizing continuous stock returns into frequency-based bins and encoding multi-stock states as immutable integer tuples, using dictionaries for the state table.

Implement a learning trader with q-learning on sp data, using apple, microsoft, amazon as features. Prepare data by downloading the data set, dropping missing values, and splitting train and test.

Discover statistical factor models through unsupervised learning with PCA, revealing hidden factors that drive stock returns beyond CAPM, including multi-factor models like Fama–French.

Derive principal component analysis and show the PCA transform, via eigen decomposition, yields an orthogonal, variance-ordered Z matrix for dimensionality reduction and uncorrelated factors.

Apply PCA to stock returns in a Colab notebook, clean data, compute log returns, standardize, and compare the first principal component to the S&P 500, highlighting the market factor.

Explore sequence modeling for stock returns, focusing on hidden Markov models to capture volatility clustering and non-normal distributions via Gaussian mixtures, not about making perfect predictions.

Explore hidden Markov model parameters for stock returns, focusing on state transitions, the state transition matrix, and gaussian means and variances for low and high volatility.

This lecture explains three core hidden Markov model tasks: compute P of X, estimate parameters via gradient descent or EM, and decode with the Viterbi algorithm, using Gaussian emissions.

Apply a hidden Markov model to identify low and high volatility regimes in stock returns, implement in Colab, infer state sequences, and visualize regime stability and transitions.