Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Software Development Tools No-Code Development
Business
Entrepreneurship Communication Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certifications Network & Security Hardware Operating Systems & Servers Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Paid Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement & Gardening Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition & Diet Yoga Mental Health Martial Arts & Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Learning Teacher Training Test Prep Other Teaching & Academics
Web Development JavaScript React Angular CSS Node.Js PHP HTML5 Vue JS
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Amazon AWS Cisco CCNA CompTIA Security+ Microsoft AZ-900
Microsoft Power BI SQL Tableau Data Modeling Business Analysis Business Intelligence MySQL Qlik Sense Data Analysis
Unity Unreal Engine Game Development Fundamentals C# 3D Game Development C++ Unreal Engine Blueprints 2D Game Development Mobile Game Development
Google Flutter iOS Development Android Development Swift React Native Dart (programming language) Kotlin Mobile App Development SwiftUI
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting Canva InDesign Character Design Procreate Digital Illustration App
Life Coach Training Personal Development Neuro-Linguistic Programming Personal Transformation Life Purpose Mindfulness Sound Therapy Coaching CBT Cognitive Behavioral Therapy
Business Fundamentals Entrepreneurship Fundamentals Freelancing Business Strategy Startup Business Plan Online Business Blogging Leadership
Digital Marketing Social Media Marketing Marketing Strategy Internet Marketing Google Analytics Copywriting Email Marketing Startup YouTube Marketing
2022-06-10T18:55:36Z

DevelopmentData ScienceNatural Language Processing

Data Science: Transformers for Natural Language Processing

BERT, GPT, Deep Learning, Machine Learning, & NLP with Hugging Face, Attention in Python, Tensorflow, PyTorch, & Keras
Highest rated
Rating: 4.8 out of 54.8 (103 ratings)
570 students
Created by Lazy Programmer Team, Lazy Programmer Inc.
Last updated 6/2022
English
English [Auto]

What you'll learn

  • Apply transformers to real-world tasks with just a few lines of code
  • Fine-tune transformers on your own datasets with transfer learning
  • Sentiment analysis, spam detection, text classification
  • NER (named entity recognition), parts-of-speech tagging
  • Build your own article spinner for SEO
  • Generate believable human-like text
  • Neural machine translation and text summarization
  • Question-answering (e.g. SQuAD)
  • Zero-shot classification
  • Understand self-attention and in-depth theory behind transformers
  • Implement transformers from scratch
  • Use transformers with both Tensorflow and PyTorch
  • Understand BERT, GPT, GPT-2, and GPT-3, and where to apply them
  • Understand encoder, decoder, and seq2seq architectures
  • Master the Hugging Face Python library

Requirements

  • Install Python, it's free!
  • Beginner and intermediate level content: Decent Python programming skills
  • Expert level content: Good understanding of CNNs and RNNs and ability to code in PyTorch or Tensorflow

Description

Hello friends!

Welcome to Data Science: Transformers for Natural Language Processing.


Ever since Transformers arrived on the scene, deep learning hasn't been the same.

  • Machine learning is able to generate text essentially indistinguishable from that created by humans

  • We've reached new state-of-the-art performance in many NLP tasks, such as machine translation, question-answering, entailment, named entity recognition, and more

  • We've created multi-modal (text and image) models that can generate amazing art using only a text prompt

  • We've solved a longstanding problem in molecular biology known as "protein structure prediction"


In this course, you will learn very practical skills for applying transformers, and if you want, detailed theory behind how transformers and attention work.

This is different from most other resources, which only cover the former.


The course is split into 3 major parts:

  1. Using Transformers

  2. Fine-Tuning Transformers

  3. Transformers In-Depth


PART 1: Using Transformers


In this section, you will learn how to use transformers which were trained for you. This costs millions of dollars to do, so it's not something you want to try by yourself!

We'll see how these prebuilt models can already be used for a wide array of tasks, including:

  • text classification (e.g. spam detection, sentiment analysis, document categorization)

  • named entity recognition

  • text summarization

  • machine translation

  • question-answering

  • generating (believable) text

  • masked language modeling (article spinning)

  • zero-shot classification

This is already very practical.

If you need to do sentiment analysis, document categorization, entity recognition, translation, summarization, etc. on documents at your workplace or for your clients - you already have the most powerful state-of-the-art models at your fingertips with very few lines of code.

One of the most amazing applications is "zero-shot classification", where you will observe that a pretrained model can categorize your documents, even without any training at all.


PART 2: Fine-Tuning Transformers


In this section, you will learn how to improve the performance of transformers on your own custom datasets. By using "transfer learning", you can leverage the millions of dollars of training that have already gone into making transformers work very well.

You'll see that you can fine-tune a transformer with relatively little work (and little cost).

We'll cover how to fine-tune transformers for the most practical tasks in the real-world, like text classification (sentiment analysis, spam detection), entity recognition, and machine translation.


PART 3: Transformers In-Depth


In this section, you will learn how transformers really work. The previous sections are nice, but a little too nice. Libraries are OK for people who just want to get the job done, but they don't work if you want to do anything new or interesting.

Let's be clear: this is very practical.

How practical, you might ask?

Well, this is where the big bucks are.

Those who have a deep understanding of these models and can do things no one has ever done before are in a position to command higher salaries and prestigious titles. Machine learning is a competitive field, and a deep understanding of how things work can be the edge you need to come out on top.

We'll also look at how to implement transformers from scratch.

As the great Richard Feynman once said, "what I cannot create, I do not understand".


SUGGESTED PREREQUISITES:

  • Decent Python coding skills

  • Deep learning with CNNs and RNNs useful but not required

  • Deep learning with Seq2Seq models useful but not required

  • For the in-depth section: understanding the theory behind CNNs, RNNs, and seq2seq is very useful


UPDATES TO LOOK FORWARD TO:

  • More fine-tuning applications

  • More in-depth conceptual lectures

  • Transformers implemented from scratch


Thank you for reading and I hope to see you soon!

Who this course is for:

  • Anyone who wants to master natural language processing (NLP)
  • Anyone who loves deep learning and wants to learn about the most powerful neural network (transformers)
  • Anyone who wants to go beyond typical beginner-only courses on Udemy

Instructors

Lazy Programmer Team
Artificial Intelligence and Machine Learning Engineer
Lazy Programmer Team
  • 4.6 Instructor Rating
  • 51,914 Reviews
  • 200,767 Students
  • 17 Courses

Today, I spend most of my time as an artificial intelligence and machine learning engineer with a focus on deep learning, although I have also been known as a data scientist, big data engineer, and full stack software engineer.

I received my first masters degree over a decade ago in computer engineering with a specialization in machine learning and pattern recognition. I received my second masters degree in statistics with applications to financial engineering.

Experience includes online advertising and digital media as both a data scientist (optimizing click and conversion rates) and big data engineer (building data processing pipelines). Some big data technologies I frequently use are Hadoop, Pig, Hive, MapReduce, and Spark.

I've created deep learning models to predict click-through rate and user behavior, as well as for image and signal processing and modeling text.

My work in recommendation systems has applied Reinforcement Learning and Collaborative Filtering, and we validated the results using A/B testing.

I have taught undergraduate and graduate students in data science, statistics, machine learning, algorithms, calculus, computer graphics, and physics for students attending universities such as Columbia University, NYU, Hunter College, and The New School.

Multiple businesses have benefitted from my web programming expertise. I do all the backend (server), frontend (HTML/JS/CSS), and operations/deployment work. Some of the technologies I've used are: Python, Ruby/Rails, PHP, Bootstrap, jQuery (Javascript), Backbone, and Angular. For storage/databases I've used MySQL, Postgres, Redis, MongoDB, and more.

Lazy Programmer Inc.
Artificial intelligence and machine learning engineer
Lazy Programmer Inc.
  • 4.6 Instructor Rating
  • 132,874 Reviews
  • 496,185 Students
  • 31 Courses

Today, I spend most of my time as an artificial intelligence and machine learning engineer with a focus on deep learning, although I have also been known as a data scientist, big data engineer, and full stack software engineer.

I received my first masters degree over a decade ago in computer engineering with a specialization in machine learning and pattern recognition. I received my second masters degree in statistics with applications to financial engineering.

Experience includes online advertising and digital media as both a data scientist (optimizing click and conversion rates) and big data engineer (building data processing pipelines). Some big data technologies I frequently use are Hadoop, Pig, Hive, MapReduce, and Spark.

I've created deep learning models to predict click-through rate and user behavior, as well as for image and signal processing and modeling text.

My work in recommendation systems has applied Reinforcement Learning and Collaborative Filtering, and we validated the results using A/B testing.

I have taught undergraduate and graduate students in data science, statistics, machine learning, algorithms, calculus, computer graphics, and physics for students attending universities such as Columbia University, NYU, Hunter College, and The New School. 

Multiple businesses have benefitted from my web programming expertise. I do all the backend (server), frontend (HTML/JS/CSS), and operations/deployment work. Some of the technologies I've used are: Python, Ruby/Rails, PHP, Bootstrap, jQuery (Javascript), Backbone, and Angular. For storage/databases I've used MySQL, Postgres, Redis, MongoDB, and more.

Top companies choose Udemy Business to build in-demand career skills.
NasdaqVolkswagenBoxNetAppEventbrite
  • Udemy Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Investors
  • Impressum Kontakt
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Accessibility statement
Udemy
© 2022 Udemy, Inc.