Deep Learning From Scratch

Explore how artificial neural networks mimic the biological brain, with neurons as basic units that receive inputs via dendrites, process information, and generate new data through generative AI.

Explore fully connected neural networks and their three-layer architecture. Learn forward and backward passes, weights, and how they model linear and nonlinear functions through training.

Explains how bias input and activation functions inject non-linearity into a simple fully connected network to model nonlinear data beyond linear sums.

the lecture presents a fully connected neural network with two inputs, a three-neuron hidden layer, and one output, using weights and biases to compute activations and learn 13 parameters.

Apply activation functions to the weighted sum in each neuron to inject non-linearity into neural networks. Without activation, a network acts like linear regression, learning only linear combinations.

Linear activation, y = x, passes inputs with unlimited range and is used at the output layer; its constant derivative provides no learning signal, so avoid it in hidden layers.

An overview of leaky ReLU, a rectified linear unit that preserves negative inputs with a small fixed slope a, where f(x)=x for x>0 and f(x)=a x for x<0; not adaptive.

Explore the sigmoid activation function, an s-shaped curve that outputs between 0 and 1, suitable for hidden or output layers, and binary classification with a 0.5 threshold.

Explore the tanh activation function, the tangent hyperbolic function, a bipolar shifted version of sigmoid with a -1 to 1 range and zero-centered data for neural networks.

Explore how a simple fully connected network uses forward and backward passes to learn by minimizing mean squared error cost function through weight and bias updates via an optimization algorithm.

Put together a simple neural network's forward pass, compute y cap, and apply backward pass with gradient descent to update weights w1, w2, and bias b, minimizing the cost function.

Distinguish multivariate from univariate functions and practice partial differentiation, treating other variables as constants to compute partial derivatives, with examples like z = x y + x.

Explain how gradients guide neural networks toward minima or maxima by differentiating functions, setting the gradient to zero, and using the first derivative test on parabolas like x^2.

Apply the chain rule to a multivariable function by substituting an intermediate variable, then differentiate with respect to that variable and with respect to the desired variable, illustrating backpropagation.

Explore backpropagation equations that update neural network weights via the gradient of the cost, using the chain rule. Relate it to the forward pass, backward pass, and the activation function.

train a simple neural network on kaggle data using python with tensorflow keras and pytorch, then set up anaconda and jupyter notebook or google colab to run it.

Explore supervised learning and distinguish classification versus regression problems, with output activation choices like linear or ReLU for regression and sigmoid or softmax for classification, including binary and multi-class scenarios.

Train a neural network in TensorFlow on the mobile price classification dataset from Kaggle, mapping mobile features like ram, memory, cores, and battery to price range label (low, medium, high).

Import pandas, numpy, and matplotlib, read the trained.csv dataset from the current directory, and inspect the first and last rows with data.head() and data.tail(), prepping for TensorFlow neural network work.

Perform exploratory data analysis on the dataset with pandas, using describe to reveal per-feature statistics like mean, std, min, and max before you feed it into a neural network.

Learn how correlation measures how inputs affect the output, identifies positive, negative, and zero relationships, and uses data to quantify each feature’s impact on price range.

Plot ram against price range with a 2d scatter plot to show that price range increases as ram increases, within the deep learning from scratch course.

Assign features and the target by selecting all columns except price range as inputs, convert to numpy arrays, and set price range as the target.

Explore how neural networks train using fit, with epochs and batch size, covering forward and backward passes, gradient descent, and updating weights to optimize the loss and accuracy.

Learn how to evaluate a trained neural network on unseen test data by converting output probabilities with softmax into class predictions using argmax, for multi-class price range classification.

Demonstrate how neural networks approximate relationships and why stock time series demand recurrent neural networks that leverage past prices to learn trends.

Understand how recurrent neural networks handle sequential data by using feedback from past outputs to influence current predictions, with hidden state h_t and current output y_t, enabling NLP and chatbots.

Learn how neural networks classify images using dog and cat examples and probability outputs, and how deep learning trains models that accept images directly with convolutional neural networks.

Discover why convolutional neural networks excel at image processing by automatically extracting features with convolutional layers and learning them via fully connected layers for robust object recognition.

Explore the cnn architecture with feature learning and classification, using convolutional layers, ReLU activations, pooling, flattening, fully connected layers, and softmax for classification.

Examine the mathematics of 2D convolution on images, including padding and stride with a 2x2 box filter, and how a 2D layer learns the best feature-extracting filters.

Apply a 2d kernel to a binary image using python libraries, illustrating convolution and kernel creation, and visualize the convolved feature map.