Web Crawling with Nodejs (H&M, Amazon, LinkedIn, AliExpress)
What you'll learn
- Differences between web crawling and web scraping in Nodejs
- The 3 main methods to use in web crawling, and when to use what method!
- How to get data from sites like H&M and AliExpress easily and fast using their hidden API's
- How to build a web crawler for server rendered sites like Amazon to crawl all their products
- How to build a Puppeteer based web crawler for a site that requires JavaScript like Linkedin
Requirements
- Basic JavaScript
Description
Do you want to build a webcrawler in Nodejs?
In this course you will learn how to build a webcrawler using the newest JavaScript syntax with popular sites like H&M, Amazon, LinkedIn and AliExpress!
You'll learn how to find hidden API's on sites like H&M and AliExpress and see how you can even avoid building a web crawler in the first place, you can save a lot of time this way!
Then I show how to build a web crawler for Amazon the test-driven way, by building out tests for the various product page layouts there is on Amazon.
After that we'll take a look at how to automate login and scraping profiles from LinkedIn using Puppeteer, the automated Chromium browser!
Who this course is for:
- Students looking to learn web crawling with Nodejs
- Students looking to learn web scraping with Nodejs
Instructor
Stefan has been building software since elementary school starting out with Visual Basic 6.0. Since then he has been dabbling with Python(Pygame), PHP & MySQL, and Java during university. Since he discovered the JavaScript world he found an intense interest in web development especially using ReactJs/React Native and using NodeJs for the backend.