Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS AWS Certified Developer - Associate CompTIA Security+
Photoshop Graphic Design Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Mindfulness Personal Development Personal Transformation Meditation Life Purpose Coaching Neuroscience
Web Development JavaScript React CSS Angular PHP WordPress Node.Js Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Retargeting
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Analysis Data Modeling Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Freelancing Blogging Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee

This course includes:

  • 7 hours on-demand video
  • 32 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
Development Web Development Node.Js

Learn Web Scraping with NodeJs in 2020 - The Crash Course

Learn and be great at Web Scraping with NodeJs and tools like: Puppeteer by Google, Request, Cheerio, NightmareJs.
Rating: 4.4 out of 54.4 (393 ratings)
2,499 students
Created by Grohs Fabian
Last updated 5/2019
English
30-Day Money-Back Guarantee

What you'll learn

  • Create Data Scrapers from Scratch to Finish with NodeJs
  • Choosing the right tools for Scraping different websites
  • How to use the Top Scraping tools for NodeJs to your Advantage
  • How to Automate User Interactions with NodeJs
  • Build Scrapers with Puppeteer by Google
  • Build Scrapers with the native Request & Cheerio
  • Learn how to scrape with NightmareJs

Course content

10 sections • 50 lectures • 7h 8m total length

  • Preview02:11
  • Preview03:29
  • Preview10:31

  • Why & When to Choose Scraping
    03:56
  • The biggest "problem" with scraping
    02:37
  • Running with Terminal / CMD & Basics on VSCode Debugger
    06:38
  • The Request library / method
    05:12
  • The Browser Automation method
    06:10

  • 01 - How to spoof / add custom request headers
    06:51
  • 02 - Deal with GZIP Compression enabled websites
    04:44
  • 03 - Selectors & More Parsing
    19:06
  • 04 - Multiple Movies & Saving JSON to File
    07:59
  • 05 - Exporting data to CSV
    06:48
  • 06 - Easiest method of getting the proper selectors
    05:36
  • 07 - Downloading Images Locally
    10:21
  • 08 - Promisifying callback based functions
    09:12

  • Using a Proxy Server
    04:43
  • Status Codes & Error Handling
    10:04
  • Manual Login with CSRF Protection & Cookie Parsing
    18:35

  • 01 - Introduction and inspecting the structures
    08:28
  • 02 - Building the scraper
    20:24
  • 03 - Parsing latest posts of an ig user
    12:08

  • 01 - Installation, introduction and testing
    04:19
  • 02 - Writing a simple automated task
    09:04
  • 03 - Generating PDF's, Emulating Phone Views, Getting Title & Url of pages
    07:36
  • 04 - Using Puppeteer to login on Instagram
    11:35
  • 05 - Making Websites Load Much Faster with a simple trick
    08:23
  • 06 - HTTP Basic Authentication
    03:46
  • 07 - Ignoring HTTPS / SSL Errors & Changing the ViewPort
    03:40
  • 08 - Learning how to use Proxies
    02:55
  • 09 - Exposing and working with Custom Functions in Puppeteer
    09:25

  • 01 - How to login on Twitter
    10:21
  • 02 - Refactoring code and improving it
    07:14
  • 03 - How to post Tweets with Puppeteer
    11:33
  • 04 - Scraping Twitter User Details
    13:56
  • 05 - Scraping More User Details
    15:45
  • 06 - Scraping User Tweets
    14:09
  • 07 - Scraping More Tweet Details ( Likes, Replies..etc )
    14:10
  • 08 - Handling The Infinite Loader of the Tweets
    08:28

  • 01 - Creating the Module Structure
    06:43
  • 02 - Parsing Product Details
    11:28
  • 03 - More Details and Tricks on the Product Details page of Amazon
    06:59

  • 01 - Installation, introduction and testing
    06:28
  • 02 - Talking and testing more about NightmareJs
    07:15
  • 03 - Scraping Ycombinator News with Vanilla Javascript
    13:02
  • 04 - Scraping Ycombinator News - More details
    10:16
  • 05 - Scraping Ycombinator News - Modulate & Pagination
    11:40
  • 06 - Handling Downloads
    06:06
  • 07 - Injecting Custom Javascript
    05:23

  • Congratulations!
    00:54

Requirements

  • JavaScript Knowledge with ES6 Syntax
  • Be familiar with CSS / jQuery selectors

Description

Get into the world of Web Scraping and Data Mining with NodeJs. Learn modern methods of scraping with NodeJs - Puppeteer and with direct NodeJs Requests.


Introduce yourself and improve your knowledge on Scraping

  • Learn Scraping with Puppeteer, NightmareJs or Manual Requests

  • Build scraper modules for various websites ( Imdb, twitter, instagram..etc )

  • Learn multiple ways of scraping and when to choose them

  • Get familiar with the ethics, do's and dont's of Scraping


Enjoy coding and learning Web Scraping with real world examples and real world problem solving while building scrapers with NodeJs.


Web Scraping is a very gray area and not many talk about it or even teach about this. You are going to find valuable scraping informations and techniques that you can directly put to practice for yourself.

I've been working with Data Mining with NodeJs for more than 2 years on dozens of websites and I've learned many ways of creating a scraper and the best practices. All of which you are going to find out and learn in just a few hours in this course.


Overview

I designed this Web Scraping Crash Course to be easily understood by absolute beginners and people who already have some knowledge about the subject. 

Complete crash course with all files and code samples, you’ll be able to work alongside with me as we work through each concept and scraper module.

This is not some random tutorial that you usually find on the internet with extremely simple examples. I am showing you everything that you need to think about when starting to build a scraper with NodeJs, while building, problem solving techniques and all you need to know  that by the end of the course to be confident and create one for yourself.

Who this course is for:

  • Complete beginners interested in learning Data Scraping with NodeJs
  • People who already have some Basic Knowledge but want to Take it to the Next Level

Instructor

Grohs Fabian
Full Stack Developer, Data Miner and Automator
Grohs Fabian
  • 4.4 Instructor Rating
  • 393 Reviews
  • 2,499 Students
  • 1 Course

Fabian is a humble self taught Web Developer and Data Miner, having a lot of experience working with Freelance clients, creating more than 20+ High Quality and Ready to Use products sold to more than 3000+ clients. 

I like you and I am ready to help you reach your programming goals.

I am ready to guide and work along with you.

Ready to take yourself and your skills to the next level?

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.