
Before you can scrape websites, you need to be able to analyse them and understand how they are put together. This quick refresher brings you up to speed. This lecture gives an overview of how a web browser knows where to go to download the data its about to show in its user interface.
There are three core types of websites you will encounter when scraping - this lecture introduces you to them as basic building blocks.
This brief lecture outlines why it is beneficial to analyse each site you want to scrape before you start coding.
In Webpage Anatomy 101, we look at what is needed to build a page, and where data comes from. This is fundamental knowledge needed in order to be able to analyse and prepare for scraping a website.
Great craftspeople need great tools! This lecture demonstrates the use of browser developer tools in Chrome at an introductory level.
Before we move onto actual code, we will look at some examples of different website page types, using the Developer tools we looked at in the previous lecture.
Moving on from the basic website type, in this lecture we look at more advanced types that you will come across
This lecture gives a deep dive into different characteristics of a dynamic website
In this lecture we look deeper into what makes up a dynamic website
Dynamic sites make heavy use of different elements to automate the webpage. This lecture looks at how this is achieved.
Having covered the basics, we now look at different element types and how they can be used on a dynamic website.
Hidden elements are really really important to know about! ... they are heavily used on interactive websites to carry information between the browser and the server - you need to pay careful attention to this section!
Here's the reason we came to this party! ... in this lecture we will write some code to GET the contents of a web-page and get it ready for data extraction by parsing.
This lecture describes the PARSE process, giving some grounding theory and also example code to get you started.
Before we finish up, lets look at what we've learned, and what lies ahead in your road to web-scraping mastery!
Web scraping is a highly paid, in demand skill
If you work with data, are a growth hacker, or starting a side hustle, you will need to start scraping data from websites sooner than you think ... the web is the worlds biggest database but only a few know how to get access to that data in an efficient way.
Starting to web-scrape by learning a framework like scrapy or beautiful soup is like trying to learn to dive before you can swim ... it looks like fun, and seems awesome, but when you land in the water, you realize you're missing something (oh, the swimming part!!) .... if you are interested in web scraping, stick with me and read on.... it'll be worth it, honestly....
Learn to web scrape from an expert - here's proof of the value:
"Finally, an instructor who knows how to teach students so that they are not lost or confused. Thank you!"
"Great course, wish I would have watched it before reading Python Scrapy documentation."
"I loved this course. It's a great introduction to web scraping! It's short, sweet and to the point!"
"Really impressed about theory and strategies to use.. goes beyond my expectation"
Web scraping (also known as scraping, scraping, harvesting and web data extraction), is both the art & craft of extracting data from websites. Data obtained from web-scraping is used for price comparison, business lead generation, and competitive analysis.
I created this course because I want to improve the quality of web-scraping information available. Most courses assume you know a lot about scraping already, but that's mostly not the case - this course doesn't try to teach you everything, the objective is to give you a solid foundation in the basics so you can start to master the art and craft of web scraping.
It doesn't matter if you use Python, Java, C# or any other language for webscraping, this short course is light on code, and tries to use imagery and animation whenever possible to get the message across. There are various frameworks that are used to assist in web scraping depending on the programming language you use. Some of these scraping libraries these include Scrapy, Beautiful Soup and others in the Python web scraping world, and JSoup and HtmlUnit for Java for example.
If you want to learn web scraping with python or any other language, don't take *any other course* until you take this one - you'll be glad you did .. trust me, when you do take your next web-scraping course, you'll be in a far stronger position to get the most from it :)