Learn PyTorch for Natural Language Processing

Name: Learn PyTorch for Natural Language Processing
Rating: 3.2 (36 reviews)

Build smart language applications with the cutting-edge field of Deep Learning with PyTorch

Created byPackt Publishing

Last updated 5/2019

English

What you'll learn

Work with Deep Learning models and architectures including layers, activations, loss functions, gradients, chain rule, forward and backward passes, and optimizers.
Apply Deep Learning architectures to solve Machine Learning problems for Structured Datasets, Computer Vision, and Natural Language Processing.
Utilize the concept of Transfer Learning by using pre-trained Deep Learning models to your own problems.
Implementing the word embedding model and using it with the Gensim toolkit.
Processing insightful information from raw data using NLP techniques with PyTorch.
Comparing and analyzing results using Attention networks to improve your project’s performance.

Course content

2 sections • 70 lectures • 4h 33m total length

The Course overview2:33
This video will give you an overview about the course.
Quick Intro to PyTorch3:02
The aim of this video is to dive into a quick introduction to PyTorch.
• Get a basic understanding of PyTorch
• Know what we can do with Pytorch
• Learn how PyTorch compares to Tensorflow and MxNet
Installation and Jupyter Notebook Setup3:13
The aim of this video is to learn the Installation and Jupyter Notebook setup for PyTorch.
• Cover PyTorch and Jupyter in VS Code
• Take a look at PyTorch Dockerfile
• Investigate PyTorch GPU support on Docker
Tensors and Basic Tensor Operations3:44
The aim of this video is to learn about the basic tensor operations.
• Create Tensors and data types
• Create Tensor math
• Work with NumPy
Advanced Tensor Operations6:48
The aim of this video is to learn about the advanced tensor operations.
• Take a look at using the GPU
• Discuss Gradients
• Perform numerical algebra with PyTorch
Loading and Saving Data3:50
We’ll take a look at loading and saving data with this video.
• Work with dataset and dataloader
• Take a look at the built in torchvision datasets
Assignment0:40
This video is your homework for day one.
• Take a look at the Question and try solving it on your own before the next lecture
Introduction to Neural Networks3:29
The aim of this video is to get introduced to neural networks.
• Explore Machine Learning
• Get a brief understanding of neural networks and deep learning
• Quick visualization of the concepts learnt
Creating a Neural Network with PyTorch Sequential2:27
The aim of this video is to learn to create a neural network with PyTorch Sequential.
• Learn the concept of Inputs
• Learn the concept of Outputs
• Link inputs and outputs with sequential layers
Activations, Loss Functions, and Gradients3:29
The aim of this video is to learn about activations, loss functions, and gradients.
• Add activations to our model
• Hook up a loss function
• Compute gradients, and know why to use them
Forward and Backward Passes3:05
The aim of this video is to learn and implement the concepts of forward and backward passes.
• Take forward passes to the model
• Backward propagate with the computing gradients in the model
• Update the model weights
Building a Network with nn.Module4:45
In this video, build a network with nn.module.
• Get in-depth understanding of inputs and outputs
• Building an object-oriented model
• Learn loop using an optimizer
Assignment0:31
This video is your homework for day two.
• Take a look at the Question and try solving it on your own before the next lecture
Loading Structured Data for Classification4:20
The aim of this video is to load structured data for classification.
• Load a CSV dataset
• Train and test the data
Preprocessing Data3:35
The aim of this video is to learn about pre-processing data.
• Handle categorical values
• Transform a Dataset
Classification, Accuracy, and the Confusion Matrix4:44
The aim of this video is to learn about classification, accuracy, and the confusion matrix.
• Build a simple classification network
• Run training loops
• Use test data with a confusion matrix
Loading Structured Data for Regression4:41
The aim of this video is to load structured data for regression.
• Discuss regression versus classification
• Encode numbers, categories, and dates
Neural Networks for Regression3:27
The aim of this video is to create neural networks for regression.
• Build a simple regression network
• Setup the training loops
• Take a look at the reporting error
Assignment0:17
This video is your homework for day three.
• Take a look at the Question and try solving it on your own before the next lecture
Convolutional Networks for Image Analysis3:26
With this video, get to study the convolutional networks for image analysis.
• Learn what makes a convolutional network
• Know why to use convolutional networks
• Explore why are convolutional networks effective on images
Convolutional Concepts: Filters, Strides, Padding, and Pooling3:56
The aim of this video is to get an in-depth understanding of the convolutional concepts.
• Learn about Filters and Strides
• Study the concept of Pooling and Padding
Implementing a Convolutional Network3:19
The aim of this video is to implement a convolutional network.
• Use torchvision to get image datasets
• Look at stacking convolutional layers
• Train and evaluate a pre-convolutional network
Visualizing Convolutional Network Layers3:41
The aim of this video is to visualize a convolutional network layers.
• Visualize tensors with matplotlib.pyplot
• Augment a network to see intermediate results
• Visualize individual image filters
Implementing an End-To-End Deep Convolutional Network3:16
The aim of this video is to implementing an end-to-end deep convolutional network.
• Base our architecture of the AlexNet
• Connect convolutional and fully-connected classifiers
• Use CUDA with PyTorch
Assignment0:19
This video is your homework for day four.
• Take a look at the Question and try solving it on your own before the next lecture
Transfer Learning and Prebuilt Models3:03
This video aims to explain about transfer learning and prebuilt models.
• Learn about transfer learning
• Know the use transfer learning
• Explore transfer learning gotchas
Deep Learning with VGG3:00
This video aims to explain about deep learning with VGG.
• Explore VGG models in PyTorch
• Learn about the input and output layers
• Study the different layer patterns
Transfer Learning with VGG4:10
This video aims to explain about transfer learning with VGG.
• Load a pre-trained model and use an alternate dataset
• Adjust the output layers
• Train and validate your network
Transfer Learning with ResNet6:27
This video aims to explain about transfer learning with ResNet.
• Compare ResNet to VGG and load a pre-trained model
• Use an alternate dataset of MNIST digits and adjust the output layers
• Train and validate the network
Assignment0:20
This video is your homework for day five.
• Take a look at the Question and try solving it on your own before the next lecture
Recurrent Networks, RNN, and LSTM, GRU4:01
This video aims to explain about recurrent networks, RNN, and LSTM, GRU.
• Take a look at Recurrent networks
• Explore LSTM networks
• Learn about GRU networks
Text Modeling with Bag-of-Words1:33
This video aims to explain about text modeling with bag-of-words approach.
• Parse words from text with spaCy
• Dive into word ordinal numbers
• Explore the sequence of words bag-of-word encodings
Sentiment Analysis with Bag-of-Words3:29
This video aims to explain about sentiment analysis with bag-of-words technique.
• Review the data
• Build a PyTorch dataset with one hot encoding
• Perform sentiment analysis – predict the rating with regression
Sentiment Analysis with Word Embeddings4:59
This video aims to explain about sentiment analysis with word embeddings.
• Learn about language models and word embeddings with spaCy
• Encode sequences with word embeddings
• Perform sentiment analysis
Assignment0:18
This video is your homework for day six.
• Take a look at the Question and try solving it on your own before the next lecture
Introduction to GANs and DCGANs4:21
This video aims to explain about GANs and DCGANs.
• Learn about classification, regression, and generation
• Learn what a GAN is and what is a DCGAN?
• Explore how to train GANs
Implementing DCGAN Model with PyTorch3:09
This video aims to explain how to implement DCGAN model with PyTorch.
• Study about Generator models
• Learn about Discriminator models
• Implementing DCGAN Model with PyTorch
Training and Evaluating DCGAN on an Image Dataset5:28
This video aims to explain how to train and evaluate DCGAN on an image dataset.
• Load up training data
• Take a look at Generation vectors and Adversarial training
• Visualize generated images
Improving Performance2:42
This video aims to explain how to improve performance.
• Custom initialization
• Perform data normalization
• Work on improving performance
Assignment1:41
This video is your homework for day seven.
• Take a look at the Question and try to solve it on your own

The Course Overview2:47
This video will give you an overview about the course.
Using Deep Learning in Natural Language Processing2:17
An introductory video. We speak about the NLP and deep learning. An intuition of what we are about to learn.
Value of natural language processing
Value of deep learning
Deep learning over traditional NLP
Functions and Features of PyTorch3:45
Why we want to work with PyTorch as our framework of choice and convincing students why PyTorch is great to learn.
PyTorch as a multipurpose library
Appreciate the ease of PyTorch
Future of PyTorch
Installing and Setting Up PyTorch2:11
In this video, let's have a look at installing and setting up PyTorch and get going with PyTorch.
Where to get PyTorch?
How to install PyTorch?
Check installation
Understanding Sentiment Analysis and NMT4:41
Understand our project statement, the solution, and why the project falls into a given NLP paradigm.
Introduction to projects
In-depths of sentiment analyzer
In-depths of NMT
NLTK and spaCy Installations2:27
Install NLTK, spaCy, and their necessary modules.
Install NLTK and spaCy
Install stopword module in NLTK
Install English model in spacy
Tokenization with NLTK6:11
We’llhave a look at the dataset, see different data loading techniques, and perform tokenization.
Look at the dataset
Look at sentence tokenization
Tokenize the entire dataset
Stop Words5:22
In this video, we will look at the stop words. Here we’ll seek the help of NLTK and spaCy to complete this task.
Explore NLTK and spaCystopwords
Define custom stopwords
Stopword removal on the entire dataset
Lemmatization4:18
In this video, we will see how we perform lemmatization and why we even need to perform it.
Understand lemmatization
Perform lemmatization on the entire dataset
Optimize lemmatization
Pipelines4:05
In this video, we’ll stitch together all the steps and practices we have seen so far so that we have a same streamlined, and maintainable piece of code.
Understand pipelines
spaCypipelines
Custom pipelines
Working with Word Embeddings4:16
Understanding the intuitions of word embeddings.
Types of word embedding
Concepts of word embedding
Setting Up and Installing gensim0:49
Installing gensim.
Testing the installations
Exploring Word Embeddings with gensim3:07
Trying out various features in gensim.
Writing code in python using gensim
Understanding the Embeddings Created3:41
Understanding conceptual relationships in corpus.
Understand word similarities
Pretrained Embeddings Using Word2vec3:09
Detailed understanding of working of Word2vec.
Setting up Word2vec
Text source of Word2vec
Importance of pre trained embeddings
Working with Recurrent Neural Network5:09
Understand the working of RNN and explore the representation.
Types of RNN
Defining of RNN
Use cases of RNN
Implementing RNN14:33
Start working with PyTorch and implement the sentiment analyzer.
Explore the architecture
Define the model
Optimizer, Loss Function, Device selection
Results with RNN9:35
Obtaining the sentiments classified by RNN.
Sorting and Padding data within the batch, and epochs for training
Preparing Data to be fed into RNN
Running Train and Evaluation modes on model
Working with LSTM5:32
Discuss the structure of LSTM and advantages of LSTMs over RNN.
LSTM Cell components
Bidirectional LSTMs
Multilayer LSTMs
Implementing LSTM6:07
Adapting to the new architecture.
Implementing to the new architecture
Structural changes in responses
Redefining the parameters
Results with LSTM4:53
Completing our Sentiment Analyzer with LSTM.
Initializing Model
Running the Epochs
Readable live results
Intro to seq2seq2:25
Explaining more complex architectures for NLP, with schematic representation.
Key components of seq2seq
Encoders and Decoders
Use cases
Installations1:07
Additional packages and libraries to be installed.
Installing Torchtext
Installing English model
Installing the German Model
Implementing seq2seq – Encoder8:21
We will obtain the data and perform the preprocessing and write our Encoder class.
Explore Torchtext
Fields and Bucket Iterators
GRU units
Implementing seq2seq – Decoder5:30
We completed the decoder with a discussion on the architecture.
Understanding the output dimensions after each step
Pulling out the target token from the decoder
Results with seq2seq7:30
Writing our seq2seq model, to tie together encoder and decoder sections.
Defining hyper-parameters and model objects.
Training and Evaluation modes for the model
Running epochs and saving model
Introduction to Attention Networks3:27
Understand the limitations of seq2seq model, and how attention networks improve the seq2seq model.
Limitations of seq2seq model
The Intuition behind the working of Attention
Features of attention networks
Implementing seq2seq – Encoder7:31
Discuss the changes in architecture and understanding the interaction of attention layer with the encoder and decoder.
Attention Layer and the Attention Vector
Defining Attention class
Tensor transformations for attention
Results with Attention Network6:25
Completing the seq2seq model with attention network.
Changes in the decoder architecture for attention layer
Tensor transformations
Running epochs and training the model
The Way Forward3:19
Thoughts on what the steps should be to improve the skills.
Improvements for the neural translation machine
Projects for Classification
Projects for seq2seq models
Test your knowledge

Requirements

Basic knowledge of machine learning concepts and Python programming is required for this course.

Description

PyTorch: written in Python, is grabbing the attention of all data science professionals due to its ease of use over other libraries and its use of dynamic computation graphs. PyTorch is a Deep Learning framework that is a boon for researchers and data scientists.

This course takes a practical approach and is filled with real-world examples to help you create your own application using PyTorch! Learn the most commonly used Deep Learning models, techniques, and algorithms through PyTorch code. Get yourself acquainted with the advanced concepts such as Transfer Learning, Natural Language Processing and implementation of Generative Adversarial Networks. Moving further you will build real-world NLP applications such as Sentiment Analyzer & advanced Neural Translation Machine.

Contents and Overview

This training program includes 2 complete courses, carefully chosen to give you the most comprehensive training possible.

The first course, PyTorch Deep Learning in 7 Days is for those who are in a hurry to get started with PyTorch. You will be introduced to the most commonly used Deep Learning models, techniques, and algorithms through PyTorch code. This course is an attempt to break the myth that Deep Learning is complicated and show you that with the right choice of tools combined with a simple and intuitive explanation of core concepts, Deep Learning is as accessible as any other application development technologies out there. It’s a journey from diving deep into the fundamentals to getting acquainted with the advanced concepts such as Transfer Learning, Natural Language Processing and implementation of Generative Adversarial Networks. By the end of the course, you will be able to build Deep Learning applications with PyTorch.

The second course, Hands-On Natural Language Processing with Pytorch you will build two complete real-world NLP applications throughout the course. The first application is a Sentiment Analyzer that analyzes data to determine whether a review is positive or negative towards a particular movie. You will then create an advanced Neural Translation Machine that is a speech translation engine, using Sequence to Sequence models with the speed and flexibility of PyTorch to translate given text into different languages. By the end of the course, you will have the skills to build your own real-world NLP models using PyTorch's Deep Learning capabilities.

About the Authors:

Will Ballard is the chief technology officer at GLG, responsible for engineering and IT. He was also responsible for the design and operation of large data centres that helped run site services for customers including Gannett, Hearst Magazines, NFL, NPR, The Washington Post, and Whole Foods. He has also held leadership roles in software development at NetSolve (now Cisco), NetSpend, and Works (now Bank of America).
Jibin Mathew is a Tech-Entrepreneur, Artificial Intelligence enthusiast and an active researcher. He has spent several years as a Software Solutions Architect, with a focus on Artificial Intelligence for the past 5 years. He has architected and built various solutions in Artificial Intelligence which includes solutions in Computer Vision, Natural Language Processing/Understanding and Data sciences, pushing the limits of computational performance and model accuracies. He is well versed with concepts in Machine learning and Deep learning and serves as a consultant for clients from Retail, Environment, Finance and Health care.

Who this course is for:

This course is for software development professionals, machine learning enthusiasts and Data Science professionals who would like to practically implement PyTorch and exploit its unique features in their Deep Learning projects.

Learn PyTorch for Natural Language Processing

What you'll learn

Explore related topics

Course content

PyTorch Deep Learning in 7 Days40 lectures • 2hr 9min

Hands-On Natural Language Processing with Pytorch30 lectures • 2hr 25min

Requirements

Description

Who this course is for: