Practical AI with Python and Reinforcement Learning

Install the free Anaconda distribution, which includes Python, launch Anaconda Navigator, and run Jupyter Notebook to set up the course environment, with the option to use other editors.

Set up a separate conda environment, activate it, and pip install key libraries like jupyter, numpy, gym, and matplotlib to prepare your reinforcement learning toolkit.

Explore numpy, the Python data science library for creating n-dimensional arrays and fast broadcasting. Create arrays, slice and index them, and apply universal functions as you prepare for exercises.

Create NumPy arrays from Python lists and built-in functions, and generate random data with NumPy. Explore array attributes, shapes, dtype, and common operations like max, min, and reshape.

Master numpy indexing and selection, from single element and slice retrieval to broadcasting across 1d and 2d arrays, plus conditional selection for data cleaning and later pandas use.

Explore numpy operations, including element-wise arithmetic and universal functions, plus summary statistics. Understand axis control on two-dimensional arrays for sum, mean, max, variance, and std.

Explore the numpy exercise notebook, practice creating arrays of zeros, ones, and fives, and avoid overwriting the example output by coding in the input cell rather than the output cell.

Explore matplotlib, the foundational Python plotting library, and learn functional versus object-oriented approaches to create and customize plots, including figures, subplots, styling, and basic data plots.

Learn matplotlib basics with the functional plt.plot approach, label axes, set axis limits, and save figures, then preview moving to the robust object oriented figure API.

Explore the Matplotlib figure object, from creating a blank canvas with plt.figure to adding axes with add_axes, sizing, positioning, and plotting on them, including insets and multiple axes.

Explore building and configuring Matplotlib figures: create a blank figure, add and customize multiple axes, plot data, and set limits, labels, and titles for clear, zoomed-in views.

Explore how plt.subplots creates a figure and a numpy axes array to arrange multiple plots. Index into the axes, plot on each subplot, and adjust spacing with tight layout.

Master Matplotlib styling by adding legends and labels to plots, then tailor colors, line widths, markers, and tickers for clearer, customizable visualizations.

Learn to customize Matplotlib visuals with colors, line styles, widths, and markers using hex colors, rgb values, and marker properties for clear, labeled plots.

Explore advanced matplotlib commands, including log scales, custom ticks, LaTeX math, axis spacing, grids, spines, and twin axes, with notes on using the gallery and Stack Overflow for reference.

Explore Matplotlib exercises: plot E equals MC squared from mass data using numpy, then compare 2007 and 2020 yield curves, adjust legends, and pursue bonus logarithmic and twin-axis plots.

Explore matplotlib plotting basics by solving exercises that generate data with numpy, plot mass versus energy (mc^2), customize legends and labels, and create subplots and twin axes.

Clarify the differences between machine learning, deep learning, reinforcement learning, and artificial intelligence, and show how supervised, unsupervised, and deep reinforcement learning combine with artificial neural networks.

Explore the supervised machine learning process, including train-test split, model evaluation, and hyperparameter tuning, using Python and scikit-learn to build data-driven predictions.

Explore the Pandas series, a one-dimensional ndarray with a labeled index, enabling data access by named keys like USA and easy creation from lists or dictionaries.

Explore Pandas series fundamentals, including named index creation and broadcasted operations aligned by labels. Learn to handle missing values with add method and fill_value, and observe dtype becoming float.

Explore creating and reading Pandas data frames from NumPy arrays and CSV files, using indices, columns, and df.info to inspect structure.

Learn how to inspect a pandas data frame: read a tips.csv, view columns and index, use head and tail, and summarize with info, describe, and transpose to understand data distribution.

Explore how to retrieve and manipulate columns in pandas data frames: select single or multiple columns, create and modify new columns, and drop columns with axis guidance.

Learn to manage pandas dataframes by setting and resetting the index, then select, drop, and append rows using iloc and loc for integer- and label-based queries.

Learn to perform the train test split, train on training, evaluate on test with mean squared error, and tune ridge hyperparameters, with notes on scaling and cross-validation prospects.

Explore the theory of artificial neural networks, from the perceptron to activation and cost functions, then implement feedforward networks with Keras and TensorFlow using backpropagation for regression and classification.

Learn how a simple perceptron models biological neurons with inputs, weights, and a bias, then scales to multi-layer perceptrons, activation functions, and backpropagation in deep learning.

Expand from a single perceptron to a multilayer perceptron with fully connected layers in a feed-forward network, where two or more hidden layers form a deep neural network.

Explore activation functions in neural networks, from z equals x times w plus b to outputs via sigmoid, tanh, and relu, and apply them to binary and multiclass classification.

Explore multi-class classification in practical ai with python and reinforcement learning, distinguishing non-exclusive and mutually exclusive classes, encoding data with one-hot/dummy variables, and choosing sigmoid or softmax activation.

Explore cost functions, including loss and error measures, and master gradient descent, back propagation, and adaptive methods like Adam to minimize prediction errors in neural networks.

Learn how backpropagation moves backward through a neural network to update weights and biases via gradient descent, starting at the final layer and propagating errors to earlier layers.

Master Keras syntax basics for TensorFlow by using a notebook, pandas, numpy, and seaborn to prepare a fake regression dataset, perform a train-test split, and scale features with min-max scaler.

Learn to build and train a Keras sequential model with dense layers and relu activations for regression, using mean squared error, x_train, y_train, and generating predictions.

Evaluate a Keras syntax basics model with a test set, computing mean absolute error, mean squared error, and root mean squared error, then compare predictions to true values.

Explore exploratory data analysis and feature engineering for a regression model that predicts house prices using King County data. Visualize distributions, correlations, and geography to identify features and outlier considerations.

Perform a train-test split, scale features with a minmax scaler, then build a keras regression model (19-input dense layers) trained with adam and mean squared error using validation data.

Evaluate a Keras regression model by comparing training and validation losses, assess with mean squared error and explained variance, and generate scaled predictions for a new house.

learn to perform a classification task with tensorflow and keras, and prevent overfitting using early stopping and dropout, with exploratory data analysis, train-test split, and preprocessing on a cancer dataset.

Develop a Keras binary classifier with dropout and early stopping to prevent overfitting, and evaluate performance using classification reports and confusion matrices.

Explore the Keras project notebook to build a binary classification model with TensorFlow 2.0 Keras API, using the Lending Club dataset, feature engineering, and exploratory data analysis.

Analyze an imbalanced loan dataset with exploratory data analysis, using heatmaps, histograms, scatter plots, and box plots to reveal feature relationships and create a loan repaid label.

Begin data preprocessing by evaluating missing data, deciding to drop or fill it, review counts, and drop employment title and employment length; examine loan status, charged off versus fully paid.

Continue data preprocessing by handling missing data, dropping the title column, and filling mortgage accounts via total accounts means with pandas. Drop remaining nulls.

Encode categorical loan data by mapping term values and applying one-hot encoding for subgrade and other categories, and extract zip codes and earliest credit year for TensorFlow-ready features.

split the data into train and test sets and normalize with a min max scaler, drop the loan status column, and define x and y with loan repaid as label.

Create and train a Keras model for binary classification using sequential and dense layers with dropout, relu activations, and a sigmoid output, compiled with binary_crossentropy and Adam.

Evaluate the trained keras model on the test set, generate predictions, and review a classification report and confusion matrix to assess precision, recall, and f1 scores.

Discover tensorboard, a visualization tool for TensorFlow, to view training dashboards, analyze models via logs, histograms, graphs, and images, using a tensorboard callback.

Explore the theory and core operations of convolutional neural networks—kernels, filters, convolutions, and pooling—and apply them to MNIST, CIFAR-10 grayscale and color images, malaria cell classification, and fashion datasets.

Explore image filters and kernels, learn how to apply a 3 by 3 kernel to images, and see how strides, padding, and convolution shape features extracted by convolutional neural networks.

Discover how convolutional layers use localized filters to reduce parameters, preserve two-dimensional image structure, and learn features from grayscale and color images via stacked three-dimensional filters.

Master pooling layers in convolutional neural networks, including max and average pooling and dropout regularization, with a 2x2 window and stride two to reduce parameters while preserving essential patterns.

Explore the MNIST dataset, its 60,000 training and 10,000 test images, 28 by 28 grayscale arrays, normalized between 0 and 1, with one-hot encoded labels for 10 classes.

Learn to load MNIST data, perform one-hot encoding with to_categorical, normalize grayscale pixels by 255, and reshape to 28x28x1 for CNN training.

Create and train a CNN for MNIST, building a sequential model with Conv2D, max pooling, flattening, and dense layers, using categorical cross entropy and early stopping.

Evaluate a trained CNN on MNIST by comparing training and validation losses and accuracies, generating classification reports and confusion matrices, and predicting single digits.

Process color cifar-10 data with 32×32 RGB images, build a two-conv-layer cnn in keras, scale inputs by 255, convert labels to categorical, and train with early stopping.

Evaluate a CNN on CIFAR-10 by plotting training versus validation metrics, interpreting loss trends, and using a confusion matrix to reveal misclassifications.

Download the cell_images.zip dataset from Google Drive, unzip it, locate the data directory, and prepare TensorFlow image data batches from directories using a Jupyter notebook.

Configure train and test paths, read real images with a read function, assess shapes, and resize malaria cell images to a consistent 130 by 130 by 3 for CNN training.

Develop a convolutional neural network for real images using a TensorFlow Keras sequential model with conv2d, pooling, dropout, and dense layers, for binary classification on 130x130 color images.

Evaluate a CNN on real image files by converting predicted probabilities to binary labels, generating a confusion matrix and classification report, and exploring threshold adjustments for precision-recall trade-offs.

Explore a cnn exercise with the fashion mnist dataset to classify 28x28 grayscale clothing images, including loading, preprocessing, modeling, training, and evaluation for around 90% accuracy.

Explore fashion mnist cnn exercise solutions, loading and preprocessing data, building a 28x28x1 cnn with conv2d, pooling, and dense softmax, then training to ~90% accuracy and evaluating with classification report.

Explore core concepts of reinforcement learning, including agents, environments, policy, rewards, and discount factors; compare deterministic versus stochastic processes and preview tabular q-learning with OpenAI Gym and Keras with TensorFlow.

Explore rewards, discount factors, and the Bellman equation within a gridworld using states and four actions to build a policy and value estimates.

Deterministic policies map a state to a single action; stochastic policies assign a probability distribution over actions, illustrating the exploration-exploitation trade-off with a grid world example.

Explore OpenAI and the OpenAI Gym library, and learn to create a Python gym environment. See an agent interact with it via random actions, laying groundwork before reinforcement learning.

Explore the history and scope of OpenAI, its Gem Python library for reinforcement learning, and milestones from GPT models to Dall-e, including the nonprofit to capped-profit shift.

Explore the OpenAI Gym documentation and learn to jump to the GitHub source for environments like Mountain Car and Atari, including ram and image inputs.

Explore OpenAI Gym basics by understanding environment features, reset and step actions, and render options. Compare ram and image-based environments using Breakout to train reinforcement learning agents.

Set up a notebook, import gym, and select an Atari Breakout environment to explore OpenAI Gym environments, then iterate with env.make, env.reset, env.step, and render in human or rgb modes.

Learn how to interact with an OpenAI Gym environment by building a simple mountain car agent, using observations, actions, seeds, and a basic reward-driven loop.