Pytorch Deep Learning

Name: Pytorch Deep Learning
Rating: 4.4 (85 reviews)

From Zero To Hero (BERT & T5)

Created by神草経知

Last updated 3/2021

English

English [Auto],Korean [Auto],

What you'll learn

Python
Pytorch
BERT
Deep Learning
Image Processing
Natural Language Processing
Neural Network
Gradient Descent
transformers
huggingface
T5

Course content

7 sections • 39 lectures • 5h 32m total length

A01_Introduction5:30
In this tutorial I'm going to be explaining the very basic concept of deep learning and going to implement neural networks by pytorch using Google’s Colab. We'll cope with FCNN, CNN, RNN, and BERT in the end.
A02 Google Colaboratory6:24
We'll change runtime type to GPU in Colab. We check if pytorch tensor on is available on CUDA.
A03 Neuron1:28
I'm going to explain what is "Deep" in Deep Learning using schematic figure of a part of brain.
A04 Perceptron5:14
I'm going to explain the basic concept of Perceptron and how to implement it in Pytorch. We'll implement "feed forward" perceptron to be able to judge if a person is fat or not fat by height and weight using that perceptron.
A05 Make Your Perceptron Trainable8:16
From lecture 5 to 9, we make our perceptron trainable, which means that our network will be able to learn from the data itself by the following step:
1) normalize data
2) change activation function
3) use loss function
4) use gradient descent method.
A06 Normalize Data5:50
Learn to normalize height and weight data with a normalization function, using mean and max, subtracting the mean and dividing by max minus mean to produce tensors.
A07 Activation Function3:33
Explore activation functions by implementing sigmoid and step functions in pytorch, visualizing their outputs from -10 to 10, and addressing overflow issues in the sigmoid calculation.
A08 Loss Function3:52
Learn how to implement binary cross-entropy loss with a sigmoid in PyTorch, including computing the dot product of X and W, reshaping outputs to match targets, and debugging shape mismatches.
A09 Gradient Descent10:59
Learn to implement gradient descent in a PyTorch perceptron by training over 300 epochs with a 2.5 learning rate, using sigmoid activation, loss backpropagation, and loss history visualization.
A10 Elegant Implementation of Gradient Descent by pytorch9:34
We will make good use of pytorch.nn module more, which is very convenient and usefule when dealing with more complicated networks.
A11 Final Project For A0:55
Define and train a neural network in PyTorch to predict BMI from height and weight, normalizing inputs and testing with unseen data.
A12 Final Project for A Explained13:33
Explore building and training a simple PyTorch neural network to predict BMI from normalized height and weight, including defining the model, loss function, Adam optimizer, and unseen data.

B01 Multi Layer Perceptron(MLP)3:05
This lecture explains how simple perceptrons fail on nonlinear problems, and how adding hidden layers enables multi-layer perceptrons to solve nonlinearly separable tasks, illustrated with xor versus linearly separable patterns.
B02 Prepare Dataset1:30
I know It's a bit confusing technique, but I did it in order to get the one-hot encoded expression by one-line of code.
For instance, suppose if we want to get the one-hot encoded expression of [0,1,2,3,2,1], we first define torch.eye(4) to get the identity matrix of size 4 which can be expressed as follows.

tensor([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]])

Then if we give a list [0,1,2,3,2,1] so as to designate the indices of the identity matrix, you'll get the one hot encoded expression as a consequence.

Please also see the code below.
import torch
x = torch.eye(4)
print(x[[0,1,2,3,2,1]])

You could also use torch.nn.functional.one_hot to get the same result actually :)

torch.nn.functional.one_hot(torch.tensor([0,1,2,3,2,1]))

If you have any question, please feel free to ask me.
B03 Define MLP Model2:33
Define a perceptron-based mlp in pytorch by creating a perceptron class that inherits the model class, implements forward, and returns log softmax outputs.
B04 Multi Layer Perceptron Training&Evaluation12:51
Train and evaluate a perceptron on binary data with binary cross-entropy with logits and Adam, then extend to an mlp with a hidden layer to solve xor.
B05 Final Project Assignment & Explanation11:10
Explain the final project by implementing and training a from-scratch perceptron and a multilayer perceptron in PyTorch, using linear layers, cross-entropy loss, and accuracy evaluation, achieving 100% accuracy.

C01 Fully Connected Neural Network(FCNN) explained4:52
Explore how fully connected neural networks handle image recognition, compare sigmoid and relu activations to mitigate vanishing gradients, and learn why flattening images loses spatial structure.
C02 FCNN LOVE Letters Image Recognition11:04
Build a four-class CNN in PyTorch to recognize 5x5 letter images in Google Colaboratory, train with cross-entropy loss and Adam, and evaluate predictions on labeled data.
C03 Final Project Assignment and Explanation9:30
Build a three-layer fully connected network and a CNN in PyTorch, train on five-by-five binary images, and perform inference using binary cross-entropy with Adam.

D01 CNN explained4:12
See how convolutional neural networks use convolutional and pooling layers, with filters and max/average pooling, to extract features and absorb misalignment in images, via Colab on 28×28 fashion dataset.
D02 Prepare Dataset5:43
Prepare a 70,000-image fashion dataset using the Portuguese Vision Library, apply transforms, and load data in mini-batches for efficient PyTorch training. Verify train and test splits.
D03 CNN Define Model11:02
Define a CNN model in PyTorch using a sequential structure with convolutional layers, pooling, dropout, and dense layers to classify grayscale 28×28 images into 10 classes.
D04 CNN Train and Evaluate Model16:54
Train a cnn by moving data and labels to the appropriate device, applying cross-entropy loss with the Adam optimizer, and evaluate progress using train and validation accuracy.
D05 CNN Make a prediction3:49
Predict fashion items from the test set with a cnn in PyTorch, handling test loader, tensor moves to device, and using argmax to classify an image like trousers.
D06 Final Project17:52
Train a PyTorch cnn to classify five-by-five grayscale images of A or B using one-hot labels and cross-entropy loss, with a custom cnn class and a 100-epoch training loop.

Recurrent Neural Network(RNN) explained3:40
Explore recurrent neural networks that use previous state to influence current output for time-dependent sequences, and apply RNNs to language, data, and handwritten digits via the Collaboratory.
E02 RNN prepare data4:11
Load and prepare the 28x28 grayscale image dataset with 0-9 labels using torch, including downloading, applying transforms, creating train and test datasets, and configuring data loaders.
E03 RNN define model8:00
Define a recurrent neural network model for classifying 28×28 digit inputs on gpu, set input size 8, sequence 28, hidden 256, 10 classes, flatten outputs, and add a final classifier.
E04 RNN Train and Evaluate Model14:03
Train and evaluate an RNN model by defining a loss function (entropy loss), and setting an optimizer with learning rate 0.001, while tracking train and test accuracy and loss.
E05 RNN Inference9:15
learn to perform rnn inference on handwritten digits by preprocessing images (grayscale, crop, resize to 28×28, invert, normalize), converting to an array, reshaping, and predicting with a pytorch rnn.

Sesame Street3:26
Discover transformer models like BERT, Ernie, Big Bird, rooted in attention is all you need, with bird base and bird large variants used for classification via Hugging Face.
F02 BERT IMDB30:08
The range of the screen is not fitted in Lecture 33 and sometimes the code I was explaining was out of the screen. In Lecture 38, you can download the BERT_TransferLearning.ipynb which includes the missing part, so you can follow the code by looking at the ipynb code before I video shoot again. Sorry for the inconvenience.
BERT Model Definition20:02
Define a bert-based classifier in PyTorch, detailing vocabulary size 33522, hidden size 768, and dropout 0.1 for binary classification with 2 outputs, and demonstrate class-weighted loss and CPU training options.
F04 BERT Model Training17:30
Train a BERT model with a supervised loop, computing loss from predictions and labels, applying backward gradients, and updating weights across batches, while evaluating on the test set.
F05 BERT Model Evaluation8:08
Explore evaluating a BERT-based model by training, selecting the best epoch, and assessing performance with predictions, a confusion matrix, and metrics like accuracy, precision, and F1-score, including transfer learning insights.
F06 BERT Model Prediction3:58
Apply a BERT model to predict text data, encoding inputs with a 256-token max length and masks, and illustrate positive versus negative predictions in a transfer learning context.
F07 BERT Final Project2:57
For the PyTorch deep learning final project, build a movie review sentiment classifier that encodes text, applies a mask, and predicts positive or negative on GPU with about 75% accuracy.

Requirements

python basic syntax
basic programming skill

Description

Pytorch&Hugginface Deep Learning Course(Colab Hands-On)

Welcome to Pytorch Deep Learning From Zero To Hero Series.

If you have already mastered the basic syntax of python and don't know what to do next, this course will be a rocket booster to skyrocket your programming skill to a business applicable level.

In this course, you will be able to master implementing deep neural network from the very beginning(simple perceptron) to BERT transfer learning/Google's T5 by using pytorch and huggingface yourself by colab. Each Section will have one assignment for you to think and code yourself.

The Agenda is below.

Agenda:

Introduction
Google Colaboratory
Neuron
Perceptron
Make Your Perceptron Trainable
Normalize Data
Activation Function
Loss Function
Gradient Descent
Elegant Pytorch Gradient Descent
Final Project
Final Project Explained
Multi Layer Perceptron(MLP)
One Hot Encoding
Prepare data for MLP
Define MLP
Train & Evaluate MLP
Final Project for MLP
FCNN Explained
FCNN LOVE Letters Classification using MLP
Final Project For FCNN
CNN Explained
CNN Prepare data(Fashion MNIST)
CNN Define Model
CNN Train&Evaluate Model
CNNInference
Final Project For CNN
RNN Explained
RNN Prepare data
RNN Define Model
RNN Train Model
RNN Inference
BERT Sesame Street
BERT Prepare Data IMDB
BERT Model definition
BERT Model Training
BERT Model Evaluation
BERT Model Prediction
BERT Final Project
T5 Prepare Data
T5 Model definition
T5 Model Training
T5 Model Evaluation
T5 Model Prediction
T5 Final Project

Let's start our journey together.

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

Who this course is for:

Beginner python developers who are curious about deep learning
Beginner python developers who are curious about pytorch
Beginner python developers who are curious about Natural Language Processing
Beginner python developers who are curious about huggingface
Python developers who are curious about implementing BERT transfer learning model
Python developers who are curious about implementing T5 generative summarization learning model

Pytorch Deep Learning

What you'll learn

Explore related topics

Course content

Introduction to Deep Learning (Neural Network) by Pytorch12 lectures • 1hr 15min

From Simple Perceptron to Multi Layer Perceptron(MLP) by pytorch5 lectures • 31min

Fully Connected Neural Network Explained3 lectures • 25min

Convolutional Neural Network (CNN)6 lectures • 1hr

Recurrent Neural Network(RNN)5 lectures • 39min

BERT7 lectures • 1hr 26min

Google's T51 lecture • 16min

Requirements

Description

Who this course is for: