Udemy
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Python Scrapy: Scrape Web Data Using Python
Rating: 4.1 out of 5(21 ratings)
242 students

Python Scrapy: Scrape Web Data Using Python

Free and open source web crawling framework, written in Python
Created byStone River eLearning
Last updated 2/2017
English
English [Auto],

What you'll learn

  • Familiar with Scrapy installation
  • Familiar with Scrapy functionality
  • Able to scrape web pages for specific data

Course content

2 sections • 10 lectures • 51m total length

  • Module Introduction1:36

    Download Source files here

  • Scrapy Introduction7:33

    Explore how Scrapy works to crawl web pages and extract links, text, images, and titles, and learn to install, configure a spider, and run crawls.

  • Text Extraction10:09

    Extract text from craigslist ads by building a Scrapy spider, setting up an seo pets project, and using XPath to pull each pet name from listing rows.

  • Scrapy Execution4:13

    Run a scrapy crawl with a spider named C-L pets, extract anchor text with XPath to capture pet titles, and save results to csv or json while debugging.

  • Scraping with CSS9:14

    The lecture builds two Scrapy spiders to scrape quotes from a site, first with XPath then with CSS selectors, outlining project setup, spider structure, and extracting author and tags.

  • Another XPath Scrape7:47

    Learn to extract quotes, authors, and tags using xpath and css selectors in scrapy, create and run a new spider, and validate fresh results.

  • Project Instructions4:58

    Apply XPath-based scraping to extract book titles and URLs from two Python pages (books and resources), then harvest resource names and URLs from consistent page structures.

  • Project Solution5:25

    Explore how to create a scrapy project, build a two-page spider, and parse item data with xpath to extract names and urls, exporting to json or csv.

  • Module Summary0:39

    Wrap up the scrappy courses by summarizing our work, including text extraction with XPath and CSS. Finish with the caption project by extracting text from two pages into an array.

Requirements

  • Basic Python
  • Some experience using Python
  • Experience installing Python libraries

Description

Scraping data from webpages can be a tedious job. But it doesn’t have to be.

With Scrapy, you can scrape using XPath or CSS. With the large number of examples from both techniques, you’re sure to find a solution that fits for you.

Whether your targeting data on a single page or multiple, Scrapy can handle the job. No matter if the data is within a list, you can scrape specific patterns right out of the list. Building up your specific Scrapy job isn't a difficult task.

Scrapy is a Python library. If you're familiar with Python, XPath or CSS, you'll feel right at home using Scrapy.

At the end of this course, you will understand:

- what Scrapy is used for
- how to install it
- how to use Scapy

In summary, you'll be able to target specific elements on a webpage, whether the element is stand along or in a list. Then you can retrieve a group of those elements or just one. This technique allows you to pull down specific types of data.

The course ends with a project to help solidify what you've learned. There is a full walk through included with the project solution.

Who this course is for:

  • IT Professionals interested in Python

Report abuse