Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Web Scraping, API ,Beautiful Soup and Pandas using Python
Rating: 4.3 out of 5(12 ratings)
70 students

Web Scraping, API ,Beautiful Soup and Pandas using Python

Scrape your first web page using Python API ,Beautiful soup and structure the data using Pandas
Created byChandramouli Jayendran
Last updated 3/2021
English
English [Auto],

What you'll learn

  • Scraping of website using API, Beautiful Soup and Pandas
  • Learn about structure of Web pages and workflow of Scraping
  • Using Python for structuring data and learning to use Jupiter notebook

Course content

3 sections • 39 lectures • 4h 3m total length

  • Introduction to API3:29

    Explore how application programming interfaces enable data collection and communication between clients and servers with web interfaces, including free and paid options, keys and ids, and currency rates.

  • Introduction to HTTP4:18

    explore the HTTP protocol, including how client–server requests and responses use headers, GET and POST methods, and status codes such as 200, 404, 400, 500, and 502.

  • Introduction to JSON2:21

    Explore JSON, JavaScript object notation, a human-readable, language-agnostic data format used in web APIs and REST responses, built from dictionaries and lists.

  • Installation of Python and Jupyter Notebook2:43

    Learn to install Python and Jupiter by visiting the installation URL, choose the latest version (3.7.4), and set up PyCharm Community Edition for coding in the browser.

  • Installation of HTML request in Python Anaconda prompt1:11

    Install request-html in Anaconda prompt to enable HTML handling in Python and Jupyter, using pip install request-html and validate installation before starting API work.

  • Get a response from API6:30

    Start and use a Jupiter notebook, then send API requests with Python's requests library to fetch euro-based currency data, including USD and Yen, and view the response text or content.

  • Get JSON Response - API2:20

    Learn to fetch data in JSON format from an API, store it with a data variable using .json, and extract specific fields such as rates, USD, and base.

  • Historical data from API5:42

    Fetches historical data from an api by building the history url, requesting data, and printing readable json; explores currency rates from 2009 to 2020.

  • Wrong request in API1:23

    Learn how to handle API errors by creating a wrong URL, observing status_code responses, and parsing JSON in a Python web scraping workflow using Beautiful Soup and Pandas.

  • Website 2: iTunes API and reading the values9:06

    Discover how to query the iTunes API with JSON, fetch Beatles data for the US, parse results with Python, and structure data in Pandas while handling errors.

  • iTunes and Pandas4:37

    Explore exporting data with pandas, building a dataframe from results, and saving it as csv in Python, while handling json-like structures and file paths for end-to-end data workflows.

  • Website 3: Edamam Registration3:14

    Sign up for Edamam to access nutrition API, obtain your application id and key, and learn how to use the free plan with 200 recipes per month.

  • Fetch API data from Edamam13:50

    Learn to fetch API data from Edamam by registering for an app id and key, constructing requests with headers, and parsing the JSON response with pandas.

  • Project: Create a currency calculator application.9:17

    Create a currency calculator by fetching exchange rates from an application programming interface, inputting base and target currencies and a quantity, then compute and display the converted amount.

Requirements

  • Basics of Python and HTML
  • Installation of Python and Jupiter notebook

Description

Scrape your first web page using Python API ,Beautiful soup and structure the data using Pandas


API Python:

  • This section help you understand the working on API and how to implement the same using Python.

  • Here we will learn how to get and post the request using API and implement the same.

  • Will create a simple currency conversion calculator using JSON.

  • We will also cover API for website which we need to sign in. We will be using the API keys and ID to login and fetch the details.

  • We will explain how to structure and export the data in CSV using Pandas.

Web Scraping:

  • This Section helps you to learn Scraping the data and storing the data in our desired Format.

  • Here we will have the data scraped and use parsing of data and store it in Pandas for reference.

  • Helps in Understanding the structure of HTML and Javascript file to parse the data.

  • 2 Projects to Scrape the data and parse them as our wish.

Beautiful Soup:

  • It is easy to learn and master. for example, if we want to extract all the links from the webpage

  • It has good comprehensive documentation which helps us to learn the things quickly.

  • It has good community support to figure out the issues that arise while we are working with this library.

Who this course is for:

  • Beginners of Python who aspire to learn about web scraping
  • People aspiring to become Data scientist

Report abuse