Machine Learning Applied to Stock & Crypto Trading - Python

Explore how machine learning, a branch of AI, uses data and algorithms to imitate learning, covering unsupervised, supervised, and reinforcement learning, clustering and dimensionality reduction for trading.

Begin with data ingestion and feature engineering, then select and train models using supervised or unsupervised methods. Assess performance, guard against overfitting, and prepare models for future predictions.

Select the right family of models for your data, train with supervised or unsupervised methods, and evaluate performance using techniques like random forest, XGBoost, clustering, and principal component analysis.

Assess model performance after training by comparing true versus predicted outcomes in binary and multiclass classification and regression, using accuracy, precision, recall, f1, roc auc, and confusion matrices.

Deploy a trained model locally or on the web, using loops, cron jobs, and cloud services; learn saving, loading, and running models with fast api.

Use Google Colab as an out-of-the-box coding environment with notebooks and code cells, install packages with pip, and compare Colab with Jupyter notebooks when local setups fail.

Learn to view and read Jupyter notebooks in Google Colab by opening the notebook from the resources folder, uploading it, and saving it with the course title, preserving the code.

Fetch stock data in Google Colab using Yahoo Finance as a pandas data reader fallback, install yfinance, and download data with start and end dates, including adjusted close.

Create and activate a conda environment for ml trading, install packages via conda or pip using provided requirements files, and prepare a code directory with notebooks.

Explore pulling crypto data from Poloniex and Binance with Python requests, parsing JSON, and building price and order book data while adapting to changing API endpoints.

Master Python basics by learning variables (integers, floats, strings, booleans), operators, and lists or arrays, then work with NumPy and PyTorch tensors in Jupyter notebooks.

Build and query Python dictionaries by storing keys and values, access prices via keys like current_price, and nest dictionaries with lists to manage stock data.

Master if statements and loops in Python to control flow, using indentation and operators like in, not, and or. Apply these to stock data with for and while loops.

Master Pandas data frame manipulation for stock data by copying frames, dropping columns, selecting rows with iloc, and creating conditional targets from a future shift while iterating over rows.

Extract close prices from a dataframe, convert to array or list, modify them (divide by two), and save or load the updated dataframe with to_csv and read_csv.

Learn to backtest in pandas, using moving averages and signals to compare returns with a buy-and-hold benchmark, and understand data extraction, feature adjustments, and price assumptions to avoid look-ahead bias.

Learn practical feature engineering and data preprocessing for regression models predicting house prices. Encode numeric data with label and one hot encoding, handle nulls, and prepare Sydney prices for modeling.

Explore unsupervised learning in finance, using hidden Markov models with Gaussian emissions, K means clustering, and principal component analysis to detect regimes and cluster assets for trading strategies.

Explore the intuition behind hidden Markov models and regime detection for finance, using states, transitions, and emissions to identify uptrend, downtrend, or sideways markets.

Explore unsupervised learning with hidden Markov models to identify market regimes using returns and range data in Python, via Jupyter notebooks or Colab, with real stock data.

Train a four-state hidden Markov model on SPI data from 2017 to 2022 to reveal hidden market regimes, then visualize states and analyze means and covariances.

Visualize market regimes with hidden states by coloring price charts according to four states, then align data arrays, plot overlays, and discuss rule-based or supervised learning strategies.

Apply hidden Markov models to stock data by structuring a dataset with returns, range, and moving averages. Split dataset into training and test sets to validate unsupervised learning in trading.

Train hidden Markov models on stock data, generate predictions for train and test sets, and embed the results in a dataframe with open, adjusted close, returns, and moving averages.

Develop a backtest with moving average signals (ma12 vs ma21) and hidden Markov model states to generate trade signals and evaluate against a benchmark.

Apply hidden Markov states to filter a moving average crossover, improve returns and Sharpe ratio to 1.46, by trading in states zero and one and avoiding state three during turmoil.

Create and organize a data folder, save the dataframe to a csv file in that folder, and prepare outputs for future steps, with the next video covering k means clustering.

Demonstrate how k-means clustering iteratively assigns data to two clusters and updates centroids, using ETF volatility and returns data to reveal optimal groupings with the NI locator for pairs trading.

Engineer features from ETF returns and volatility using percent-change means and std-based volatility, scale with StandardScaler, and apply K-means clustering to group similar assets.

Apply k-means clustering to financial data, determine the optimum number of clusters using inertia and elbow method, and visualize clusters of ETFs by mean return and volatility.

Learn how to identify co integration pairs using statistical arbitrage for mean-reverting spreads in pairs trading, including computing hedge ratios, z-scores, and Python implementation.

Visualize co-integrated assets with a tsne plot to reveal their relationships in a 2d space. Filter assets by co-integration and interpret the resulting clusters for insights.

Select pairs via k-means clustering and co-integration, then compute the spread with a hedge ratio and monitor the z-score to signal mean-reverting trades.

Reduce dimensionality by transforming many features into principal components, retaining about 80% of information. Use about eight components to reveal correlations and support supervised learning, though interpretation remains limited.

Import and setup unsupervised learning tools, perform data extraction from the VIX via Yahoo Finance, and apply feature engineering with TA indicators to prepare for principal component analysis.

Apply PCA-inspired data preprocessing to stock and crypto data by turning non stationary features into stationary using percentage change, detected via p-values and t-tests.

Learn how to preprocess stock and crypto data, set a supervised learning target, apply scaling, and perform a train-test split before applying PCA for dimensionality reduction.

Apply principal component analysis to reduce 80 features to a few components, preserve about 80% of the information, and visualize the results to gauge predictability for stock and crypto trading.

Leverage unsupervised learning with hidden Markov models, K means clustering, and principal component analysis to uncover hidden regimes and identify co-integrated assets for statistical arbitrage or pairs trading.

Explore the theory behind random forests and XGBoost, including bagging versus boosting, bootstrapping, feature selection, performance evaluation, and strategies to prevent overfitting for stock and crypto trading data.

Set up a supervised machine learning workflow for Bitcoin volatility trading, ingest btc usd data, compute returns and range, perform data preprocessing, and begin feature engineering for move-based strategies.

Learn feature expansion for stock and crypto data using RSI, 12- and 21-day moving averages, day of week, rolling returns and range, plus time-step features for XGBoost preprocessing.

Enforce stationarity by converting open, high, low, and volume to percent change, assess ML readiness, and handle not a number, infinite values, and non-numeric objects before saving for XGBoost.

Apply XGBoost to perform feature selection and data preprocessing for predicting bitcoin's price move, using train-test split, cross-validation, and hyperparameter tuning.

Train a classification model with xgb, set and tune hyperparameters, and fit on the training data while evaluating using cross-validation to assess accuracy and precision, and explore feature importance.

Evaluate model performance to detect overfitting by comparing training and test results, using a time-series train-test split with no shuffle for robust out-of-sample evaluation of the xgboost binary classifier.

Build and fit an XGBoost classifier with defined hyperparameters, generate predictions and probabilities, and evaluate performance using cross-validated metrics to discuss overfitting and evaluation.

Assess a binary xgboost model's performance by tracking training and validation log loss, error, and roc curves, and tune hyperparameters to balance overfitting and predictive accuracy.

Dive into performance evaluation of an XGBoost model with k-fold cross-validation, comparing training and testing results, and reporting accuracy, precision, standard deviation, and confusion matrix insights.

Evaluate training and test confusion matrices to quantify true positives, true negatives, false positives, false negatives, and recall, and propose improvements with data shuffling and hidden Markov model regimes.

Explore evaluating an XGBoost model for stock and crypto trading by visualizing feature importance and the decision tree, tuning hyperparameters, and assessing precision and ROC performance for informed predictions.

Explore how a neural network uses input features, weights, biases, and activation functions to produce predictions through hidden and output layers, with loss and backpropagation tuning weights.

Learn how to build a PyTorch neural network for stock and crypto trading, focusing on data extraction, feature engineering, preprocessing, and training to predict the one-day-ahead spread direction.

Split the data into X and y, apply a standard scaler to features, and prepare the neural network inputs by using log returns while noting asset prices and returns.

Build a PyTorch neural network with three fully connected layers and sigmoid output for binary classification, and prepare a dataset X and Y as torch tensors with a train loader.

Set up and train a neural network with hyperparameters, use binary cross-entropy loss and AdamW optimizer, then run a forward loop to monitor accuracy and loss.