Natural Language Processing with Deep Learning in Python
4.6 (2,960 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
23,551 students enrolled

Natural Language Processing with Deep Learning in Python

Complete guide on deriving and implementing word2vec, GloVe, word embeddings, and sentiment analysis with recursive nets
Bestseller
4.6 (2,960 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
23,551 students enrolled
Last updated 10/2018
English
English [Auto-generated], French [Auto-generated], 8 more
  • German [Auto-generated]
  • Indonesian [Auto-generated]
  • Italian [Auto-generated]
  • Polish [Auto-generated]
  • Portuguese [Auto-generated]
  • Romanian [Auto-generated]
  • Spanish [Auto-generated]
  • Thai [Auto-generated]
Current price: $9.99 Original price: $199.99 Discount: 95% off
30-Day Money-Back Guarantee
This course includes
  • 13 hours on-demand video
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to Udemy's top 3,000+ courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Understand and implement word2vec
  • Understand the CBOW method in word2vec

  • Understand the skip-gram method in word2vec

  • Understand the negative sampling optimization in word2vec
  • Understand and implement GloVe using gradient descent and alternating least squares
  • Use recurrent neural networks for parts-of-speech tagging
  • Use recurrent neural networks for named entity recognition
  • Understand and implement recursive neural networks for sentiment analysis
  • Understand and implement recursive neural tensor networks for sentiment analysis
Requirements
  • Install Numpy, Matplotlib, Sci-Kit Learn, Theano, and TensorFlow (should be extremely easy by now)
  • Understand backpropagation and gradient descent, be able to derive and code the equations on your own
  • Code a recurrent neural network from basic primitives in Theano (or Tensorflow), especially the scan function
  • Code a feedforward neural network in Theano (or Tensorflow)
  • Helpful to have experience with tree algorithms
Description

In this course we are going to look at advanced NLP.

Previously, you learned about some of the basics, like how many NLP problems are just regular machine learning and data science problems in disguise, and simple, practical methods like bag-of-words and term-document matrices.

These allowed us to do some pretty cool things, like detect spam emails, write poetry, spin articles, and group together similar words.

In this course I’m going to show you how to do even more awesome things. We’ll learn not just 1, but 4 new architectures in this course.

First up is word2vec.

In this course, I’m going to show you exactly how word2vec works, from theory to implementation, and you’ll see that it’s merely the application of skills you already know.

Word2vec is interesting because it magically maps words to a vector space where you can find analogies, like:

  • king - man = queen - woman

  • France - Paris = England - London

  • December - Novemeber = July - June


We are also going to look at the GloVe method, which also finds word vectors, but uses a technique called matrix factorization, which is a popular algorithm for recommender systems.

Amazingly, the word vectors produced by GLoVe are just as good as the ones produced by word2vec, and it’s way easier to train.

We will also look at some classical NLP problems, like parts-of-speech tagging and named entity recognition, and use recurrent neural networks to solve them. You’ll see that just about any problem can be solved using neural networks, but you’ll also learn the dangers of having too much complexity.

Lastly, you’ll learn about recursive neural networks, which finally help us solve the problem of negation in sentiment analysis. Recursive neural networks exploit the fact that sentences have a tree structure, and we can finally get away from naively using bag-of-words.

All of the materials required for this course can be downloaded and installed for FREE. We will do most of our work in Numpy, Matplotlib, and Theano. I am always available to answer your questions and help you along your data science journey.

This course focuses on "how to build and understand", not just "how to use". Anyone can learn to use an API in 15 minutes after reading some documentation. It's not about "remembering facts", it's about "seeing for yourself" via experimentation. It will teach you how to visualize what's happening in the model internally. If you want more than just a superficial look at machine learning models, this course is for you.

See you in class!



TIPS (for getting through the course):

  • Watch it at 2x.

  • Take handwritten notes. This will drastically increase your ability to retain the information.

  • Write down the equations. If you don't, I guarantee it will just look like gibberish.

  • Ask lots of questions on the discussion board. The more the better!

  • Realize that most exercises will take you days or weeks to complete.

  • Write code yourself, don't just sit there and look at my code.


HARD PREREQUISITES / KNOWLEDGE YOU ARE ASSUMED TO HAVE:

  • calculus

  • linear algebra

  • probability (conditional and joint distributions)

  • Python coding: if/else, loops, lists, dicts, sets

  • Numpy coding: matrix and vector operations, loading a CSV file

  • neural networks and backpropagation, be able to derive and code gradient descent algorithms on your own

  • Can write a feedforward neural network in Theano and TensorFlow

  • Can write a recurrent neural network / LSTM / GRU in Theano and TensorFlow from basic primitives, especially the scan function

  • Helpful to have experience with tree algorithms


WHAT ORDER SHOULD I TAKE YOUR COURSES IN?:

  • Check out the lecture "What order should I take your courses in?" (available in the Appendix of any of my courses, including the free Numpy course)


Who this course is for:
  • Students and professionals who want to create word vector representations for various NLP tasks
  • Students and professionals who are interested in state-of-the-art neural network architectures like recursive neural networks
  • SHOULD NOT: Anyone who is not comfortable with the prerequisites.
Course content
Expand all 100 lectures 12:48:00
+ Outline, Review, and Logistical Things
4 lectures 16:19
Where to get the code / data for this course
02:32
How to Succeed in this Course
03:13
Tensorflow or Theano - Your Choice!
04:09
+ Beginner's Corner: Working with Word Vectors
8 lectures 55:22
What are vectors?
07:56
What is a word analogy?
07:58
Trying to find and assess word vectors using TF-IDF and t-SNE
07:42
Pretrained word vectors from GloVe
11:05
Pretrained word vectors from word2vec
06:31
Text Classification with word vectors
04:24
Text Classification in Code
06:14
Using pretrained vectors later in the course
03:32
+ Review of Language Modeling and Neural Networks
10 lectures 01:22:40
Review Section Intro
03:13
Bigrams and Language Models
14:47
Bigrams in Code
14:19
Neural Bigram Model
07:56
Neural Bigram Model in Code
06:48
Neural Network Bigram Model
09:13
Neural Network Bigram Model in Code
03:31
Improving Efficiency
14:35
Improving Efficiency in Code
04:52
Review Section Summary
03:26
+ Word Embeddings and Word2Vec
15 lectures 01:34:02
Return of the Bigram
03:07
CBOW
07:39
Skip-Gram
04:00
Hierarchical Softmax
08:22
Negative Sampling
14:11
Negative Sampling - Important Details
05:09
Why do I have 2 word embedding matrices and what do I do with them?
02:16
Word2Vec implementation tricks
04:49
Word2Vec implementation outline
04:09
Word2Vec in Code with Numpy
10:47
Word2Vec Tensorflow Implementation Details
03:58
Word2Vec Tensorflow in Code
04:06
How to update only part of a Theano shared variable
05:29
Word2Vec in Code with Theano
09:57
Alternative to Wikipedia Data: Brown Corpus
06:03
+ Word Embeddings using GloVe
14 lectures 01:43:46
GloVe Section Introduction
02:19
Matrix Factorization for Recommender Systems - Basic Concepts
21:08
Matrix Factorization Training
08:11
Expanding the Matrix Factorization Model
09:23
Regularization for Matrix Factorization
06:18
GloVe - Global Vectors for Word Representation
04:12
Recap of ways to train GloVe
02:31
GloVe in Code - Numpy Gradient Descent
16:48
GloVe in Code - Alternating Least Squares
04:42
GloVe in Code - Theano Gradient Descent
03:50
GloVe in Tensorflow with Gradient Descent
07:03
Visualizing country analogies with t-SNE
04:24
Hyperparameter Challenge
02:19
Training GloVe with SVD (Singular Value Decomposition)
10:38
+ Unifying Word2Vec and GloVe
2 lectures 19:27
Pointwise Mutual Information - Word2Vec as Matrix Factorization
12:06
PMI in Code
07:21
+ Using Neural Networks to Solve NLP Problems
13 lectures 01:21:24
Parts-of-Speech (POS) Tagging
05:00
How can neural networks be used to solve POS tagging?
04:08
Parts-of-Speech Tagging Baseline
15:18
Parts-of-Speech Tagging Recurrent Neural Network in Theano
13:05
Parts-of-Speech Tagging Recurrent Neural Network in Tensorflow
12:17
How does an HMM solve POS tagging?
07:57
Parts-of-Speech Tagging Hidden Markov Model (HMM)
05:58
Named Entity Recognition (NER)
03:01
Comparing NER and POS tagging
02:01
Named Entity Recognition Baseline
05:54
Named Entity Recognition RNN in Theano
02:19
Named Entity Recognition RNN in Tensorflow
02:13
Hyperparameter Challenge II
02:13
+ Recursive Neural Networks (Tree Neural Networks)
11 lectures 01:29:27
Recursive Neural Networks Section Introduction
07:14
Data Description for Recursive Neural Networks
06:52
What are Recursive Neural Networks / Tree Neural Networks (TNNs)?
05:41
Building a TNN with Recursion
04:47
Trees to Sequences
06:38
Recursive Neural Network in Theano
18:34
Recursive Neural Tensor Networks
06:22
RNTN in Tensorflow (Tips)
12:19
RNTN in Tensorflow (Code)
11:19
Recursive Neural Network in TensorFlow with Recursion
04:12
+ Theano and Tensorflow Basics Review
4 lectures 34:14
(Review) Theano Basics
07:47
(Review) Theano Neural Network in Code
09:17
(Review) Tensorflow Basics
07:27
(Review) Tensorflow Neural Network in Code
09:43
+ Legacy Word2vec Lectures
5 lectures 33:38
(Legacy) What is a word embedding?
09:59
(Legacy) Using pre-trained word embeddings
02:17
(Legacy) Word analogies using word embeddings
03:51
(Legacy) TF-IDF and t-SNE experiment
12:24
(Legacy) Word2Vec introduction
05:07