Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Natural Language Processing: NLP With Transformers in Python

Name: Natural Language Processing: NLP With Transformers in Python
Rating: 4.5 (3715 reviews)

Learn next-generation NLP with transformers for sentiment analysis, Q&A, similarity search, NER, and more

Created byJames Briggs

Last updated 8/2022

English

German [Auto],English [Auto],

What you'll learn

Industry standard NLP using transformer models
Build full-stack question-answering transformer models
Perform sentiment analysis with transformers models in PyTorch and TensorFlow
Advanced search technologies like Elasticsearch and Facebook AI Similarity Search (FAISS)
Create fine-tuned transformers models for specialized use-cases
Measure performance of language models using advanced metrics like ROUGE
Vector building techniques like BM25 or dense passage retrievers (DPR)
An overview of recent developments in NLP
Understand attention and other key components of transformers
Learn about key transformers models such as BERT
Preprocess text data for NLP
Named entity recognition (NER) using spaCy and transformers
Fine-tune language classification models

Course content

14 sections • 104 lectures • 11h 30m total length

Introduction2:25
A brief introduction to the course, and how to get the most out of it.
Course Overview6:33
An overview of everything we'll be covering in this course.
Hello! and Further Resources2:44
Meet transformer-based NLP in Python and explore practical applications like semantic search and vector-based recommendations, with additional resources and related courses.
Environment Setup6:13
How to setup a local Python environment that aligns to the environment used throughout the course.
Alternative Local Setup1:02
Alternative Colab Setup1:52
Learn how to setup a persistent Python environment in Google Colab.
CUDA Setup3:16
How to setup CUDA for CUDA enabled GPUs.
Apple Silicon Setup0:37

The Three Eras of AI6:58
Pros and Cons of Neural AI4:44
Explore the rise of neural networks, from deep learning to data-hungry models, and weigh adaptability and accuracy against brittleness and interpretability in natural language processing.
Word Vectors4:52
Recurrent Neural Networks4:01
Explore recurrent neural networks and their recurrence for language, with time steps and output state preserving word order, then confront vanishing and exploding gradients in training.
Long Short-Term Memory1:56
Encoder-Decoder Attention5:38
Self-Attention4:03
Multi-head Attention2:41
Positional Encoding8:58
Transformer Heads9:40
Explore how transformer heads customize bert for masked language modeling, classification, and question answering, using mask tokens, linear mappings to vocab or label spaces, softmax, and argmax.

Stopwords5:39
Here we'll start with our first NLP preprocessing technique, how we can use stopwords.
Tokens Introduction6:53
In the first part of exploration of tokens in NLP, we'll look at word, character, punctuation, part-of-word tokens and more.
Model-Specific Special Tokens5:44
In the second part of tokens in NLP, we'll look at model-specific special tokens.
Stemming5:27
We take a look and the Porter and Lancaster stemmers.
Lemmatization3:40
Here we take a look at reducing words to their lemma roots.
Unicode Normalization - Canonical and Compatibility Equivalence5:22
Here we will introduce Unicode Normalization and the two forms of equivalence, canonical and compatibility.
Unicode Normalization - Composition and Decomposition4:32
Here we take a look at the two different directions in Unicode Normalization, composition, and decomposition.
Unicode Normalization - NFD and NFC5:09
We'll move onto applying Unicode Normalization in Python with both NFD and NFC forms.
Unicode Normalization - NFKD and NFKC6:51
In the final Unicode Normalization session, we'll learn about and implement NFKD and NFKC forms.

Attention Introduction2:16
Explore four attention mechanisms—encoder-decoder attention, self-attention, bidirectional attention, and multi-head attention—and learn how they work and where they’re used in transformer models.
Alignment With Dot-Product11:59
Explore how the dot product computes alignment in encoder-decoder attention by comparing word vectors, visualized via heatmaps and 3d plots, with cosine similarity discussed as an alternative.
Dot-Product Attention4:25
Explore dot-product attention with query, key, and value tensors from embeddings and linear layers for the encoder and decoder, using softmax to produce the final attention.
Self Attention5:39
Bidirectional Attention2:23
Multi-head and Scaled Dot-Product Attention6:04
Explore multi-head attention in bidirectional self-attention within an encoder, using three heads, key/value/query tensors, and a linear projection to scaled dot-product attention.

Project Overview2:56
Getting the Data (Kaggle API)7:17
Preprocessing13:42
Building a Dataset5:31
Dataset Shuffle, Batch, Split, and Save6:42
Shuffle and batch the dataset with batch size 16, drop leftovers, split 90/10 into train and validation sets with take and skip, then save with tf.data.experimental.save.
Build and Save13:00
Initialize a Bert base cased model with TF auto model, feed input ids and attention masks, add a classifier head for sentiment, train for three epochs, and save the model.
Loading and Prediction10:39

Introduction to spaCy7:46
Extracting Entities6:22
NER Walkthrough
Authenticating With The Reddit API8:03
Pulling Data With The Reddit API12:44
Pulls reddit posts via the Reader API, authenticates, requests up to 100 posts, and builds a pandas dataframe with subreddit, name, created utc, title, self text, and score.
Extracting ORGs From Reddit Data6:24
Getting Entity Frequency3:53
Extract organization mentions from a dataframe, flatten nested lists, and count frequencies with collections.Counter to identify the top ten most frequently mentioned organizations.
Entity Blacklist3:41
NER With Sentiment19:20
NER With roBERTa8:36

Open Domain and Reading Comprehension3:18
An introduction to the two modes of Q&A, open domain (OD) and reading comprehension (RC).
Retrievers, Readers, and Generators6:25
An introduction to the three key model types we will be using in Q&A, retrievers, readers, and generators.
Intro to SQuAD 2.05:38
We introduce the SQuAD Q&A dataset.
Processing SQuAD Training Data6:53
How we process the SQuAD data to be more friendly for our use-case.
(Optional) Processing SQuAD Training Data with Match-Case4:20
We take a look at refactoring our SQuAD processing data using pattern matching syntax introduced in Python 3.10.
Processing SQuAD Dev Data
Our First Q&A Model8:36
We put together our first Q&A model.

Q&A Performance With Exact Match (EM)4:51
Introducing the ROUGE Metric4:20
We introduce the ROUGE metrics.
ROUGE in Python4:36
Learn how to implement ROUGE scores using Python.
Applying ROUGE to Q&A8:15
We take a look at applying ROUGE performance metrics to our first Q&A model.
Recall, Precision and F14:54
We work through the intuition and mathematics behind ROUGE-N.
Longest Common Subsequence (LCS)3:10
We work through the intuition and mathematics behind ROUGE-L.

Requirements

Knowledge of Python
Experience in data science a plus
Experience in NLP a plus

Description

Transformer models are the de-facto standard in modern NLP. They have proven themselves as the most expressive, powerful models for language by a large margin, beating all major language-based benchmarks time and time again.

In this course, we cover everything you need to get started with building cutting-edge performance NLP applications using transformer models like Google AI's BERT, or Facebook AI's DPR.

We cover several key NLP frameworks including:

HuggingFace's Transformers
TensorFlow 2
PyTorch
spaCy
NLTK
Flair

And learn how to apply transformers to some of the most popular NLP use-cases:

Language classification/sentiment analysis
Named entity recognition (NER)
Question and Answering
Similarity/comparative learning

Throughout each of these use-cases we work through a variety of examples to ensure that what, how, and why transformers are so important. Alongside these sections we also work through two full-size NLP projects, one for sentiment analysis of financial Reddit data, and another covering a fully-fledged open domain question-answering application.

All of this is supported by several other sections that encourage us to learn how to better design, implement, and measure the performance of our models, such as:

History of NLP and where transformers come from
Common preprocessing techniques for NLP
The theory behind transformers
How to fine-tune transformers

We cover all this and more, I look forward to seeing you in the course!

Who this course is for:

Aspiring data scientists and ML engineers interested in NLP
Practitioners looking to upgrade their skills
Developers looking to implement NLP solutions
Data scientist
Machine Learning Engineer
Python Developers

Natural Language Processing: NLP With Transformers in Python

What you'll learn

Explore related topics

Course content

Introduction8 lectures • 25min

NLP and Transformers10 lectures • 54min

Preprocessing for NLP9 lectures • 49min

Attention6 lectures • 33min

Language Classification5 lectures • 37min

[Project] Sentiment Model With TensorFlow and Transformers7 lectures • 1hr

Long Text Classification With BERT2 lectures • 38min

Named Entity Recognition (NER)9 lectures • 1hr 17min

Question and Answering6 lectures • 35min

Metrics For Language6 lectures • 30min

Requirements

Description

Who this course is for: