Building your own Neural Network from Scratch with Python

Install Python by downloading a version (3.9 or 3.8) and add it to your path, then use pip to install NumPy and Jupyter Notebook and start a Python 3 file.

Explore Python basics by creating variables with dynamic types and using numbers, strings, lists, and dictionaries. Learn to apply arithmetic, comparison, assignment, and slicing with print formatting and string methods.

Explore how Python conditional statements use if, elif, and else with proper syntax and operators. Use practical examples like sports choices to determine health outcomes, then transition toward loops.

Master Python loops by comparing while and for loops, using range, indexing, and lists; explore break, continue, and pass to control flow and real-world iterations.

Explore how to define functions, pass positional and keyword arguments, apply default values, and handle variable arguments, then return results and distinguish local and global variables.

Demonstrate object oriented programming with a smiley class, defining attributes and methods, and show how self and a constructor give each object its own identity.

Explore operator overloading in Python by treating strings, lists, and dictionaries as classes, using + and == through special methods like __add__ and __eq__, and making objects callable with __call__.

Explore Python class methods, static methods, and instance methods, using self, decorators like @staticmethod, and practical examples of manipulating and comparing object attributes.

Explore inheritance in object oriented programming through square and smiley examples. Use the super keyword and constructors to extend a superclass into subclasses with multilevel inheritance.

Explore encapsulation by protecting class attributes with private access using double underscores and setters to modify values internally, preventing external modifications.

Learn how polymorphism uses a single interface to compute areas for different objects, enabling the same code to handle various shapes while remaining easy to modify and maintain.

Explore how decorators in Python wrap functions to perform automatic authentication, enabling reusable endpoints like get_id, get_status, and get_user_name without repeating code.

Discover how generators yield exponentials on the fly, avoiding full data loads. Gain memory-efficient streaming and real-time data processing.

Install numpy and learn to create 1d and 2d arrays, inspect shapes, and perform elementwise and matrix operations, including concatenation and slicing.

Create and customize basic plots in Matplotlib by plotting x versus y, adjusting color and labels, and arrange two plots using subplots and figure–axes layouts for exponential and logarithmic graphs.

Explore the machine learning development life cycle by identifying tasks such as regression, classification, density estimation, clustering, anomaly detection, and recommendations, and selecting appropriate models.

Prioritize collecting representative, balanced data that matches the deployment environment to drive model performance. Normalize inputs, apply spectrograms for audio, and use one-hot encoding for outputs.

Explains selecting and training linear and logistic regression models with multiple inputs, using weights and bias, and when to add nonlinear features like squared terms or a sigmoid.

Learn how to select and compute loss functions for neural networks, including mean square error, mean absolute error, huber loss, and binary cross entropy, and apply likelihood-based training.

Explore gradient descent, learning rate, and loss minimization in linear regression. Compute partial derivatives and update parameters to minimize mean squared error.

Explore logistic regression with a sigmoid output, derive weight updates via partial derivatives, and compare batch, mini-batch, and stochastic gradient descent with learning rate scheduling.

Learn to apply linear regression to predict x1 from x2 using a real dataset, compare the normal equation and gradient descent, pre-process by scaling, and visualize the best-fit line.

Practice logistic regression with a sigmoid output and binary cross-entropy loss, using gradient descent and normalization to predict pass/fail from exam scores.

Explore momentum optimization, including velocity updates, epsilon and learning rate, and compare adaptive optimizers like Adagrad, RMSProp, and Adam for faster loss minimization.

Explore performance measurement in machine learning by comparing accuracy, precision, recall, specificity, and the confusion matrix. Learn to use ROC curves and AUC to evaluate and compare models.

Shuffle the data and split into training, validation, and test sets to fairly assess model performance on unseen data, select the best model, and ensure generalization in deep learning.

Explore multiclass letter recognition using a UCI dataset with 26 classes and 16 features, and compare data augmentation and feature engineering to deep learning where pixels are features.

Explore the shift from binary to multiclass classification, contrasting sigmoid with softmax, detailing multiple outputs, parameter counts, and bias handling for probabilistic class predictions.

Delve into multi-class logistic regression, probability distributions, and categorical cross-entropy loss with softmax, using maximum likelihood estimation to derive class probabilities across outputs.

Train a neural network from scratch using gradient descent to optimize multiclass classification with softmax and cross-entropy loss, employing one-hot targets for a 17-input, 26-class alphabet dataset.

Assess networks' performance using accuracy on training and validation data, observe overfitting, and adjust learning rate and momentum; conclude that CNNs and multilayer networks can reach accuracy on image data.

Explore why neural networks surpass logistic regression, learn how hidden layers and weights create complex nonlinear functions, and compare activation functions like sigmoid, tanh, ReLU, and softmax in modeling.

Learn to define and minimize losses to train neural networks, using mean square error, mean absolute error, and huber losses for regression, and binary or categorical cross entropy for classification.

Learn automatic differentiation and backpropagation to train neural networks, from forward propagation through linear layers to gradient descent optimization with activation functions and loss.

Train a neural network from scratch by building linear layers, performing forward and backward passes, computing loss, and updating weights with gradient descent using sigmoid and softmax activations.

Measure neural network performance using accuracy by evaluating predicted outputs against targets, and monitor training accuracy and loss while saving and reloading model weights.

Explore validation and testing of a neural network on multiclass classification, analyzing training versus validation loss and accuracy, and strategies to address underfitting by adjusting epochs and hidden units.

Learn to diagnose overfitting and underfitting in neural networks, compare training and validation losses, and apply data augmentation, early stopping, dropout, and L1/L2 regularization to improve generalization.

Shuffle data before splitting into training, validation, and test sets, and shuffle each epoch to prevent bias and ensure every data point influences weight optimization.

Explore the ensemble technique by training multiple models on the same data and averaging their outputs to form an ensemble prediction, with weighted averages based on model performance.

Avoid zero initialization by using random, zero-mean weight initialization. Xavier initialization sets weight variance to 1 divided by the number of neurons (considering the previous and next layer sizes).

Learn how to handle data imbalance in neural networks by using weighted categorical cross-entropy, class weights derived from total and per-class counts, and embeddings to balance training.

Begin training with a relatively high learning rate to converge quickly, then decay the learning rate as training progresses to reach better minima and improve model performance.

Learn how normalization stabilizes training by centering outputs to zero mean and unit variance, and compare batch normalization, layer normalization, and instance normalization across channels and batches.

Explore how to tune hyperparameters in deep learning, including learning rate, activation functions (sigmoid, ReLU, tanh), network depth and width, epochs, regularization, momentum, embedding, and batch size, with iterative evaluation.

build image classification models on a 5000-image, 20-class clothing dataset to predict class labels and whether clothing is for kids, using pixel inputs and train, validate, test splits.