Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS AWS Certified Developer - Associate CompTIA Security+
Photoshop Graphic Design Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Mindfulness Personal Development Personal Transformation Meditation Life Purpose Coaching Neuroscience
Web Development JavaScript React CSS Angular PHP Node.Js WordPress Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Retargeting
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Analysis Data Modeling Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Freelancing Blogging Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee

This course includes:

  • 8 hours on-demand video
  • 10 articles
  • 19 downloadable resources
  • 1 coding exercise
  • Full lifetime access
  • Access on mobile and TV
  • Assignments
Development Programming Languages Python

Web Scraping with Python: BeautifulSoup, Requests & Selenium

Web Scraping and Crawling with Python: Beautiful Soup, Requests & Selenium
Rating: 4.3 out of 54.3 (828 ratings)
5,735 students
Created by GoTrained Academy, Waqar Ahmed
Last updated 12/2018
English
English, Italian [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • Python Refresher: Review of Data Structures, Conditionals, File Handling
  • How Websites are Hosted on Servers; Basic Calls to Server (GET, POST Methods)
  • Web Scraping with Python Beautiful Soup and Requests
  • Using Selenium to handle JavaScript and AJAX
  • Diverse Web Scraping Exercises
  • Source codes (*.py files) for all Exercises can be downloaded
  • Q&A board to send your questions and get them answered quickly

Course content

19 sections • 72 lectures • 7h 55m total length

  • Preview04:22

  • Lists
    06:39
  • Dictionaries
    09:11
  • Tuples
    07:56
  • List Comprehensions - Part 1
    07:06
  • List Comprehensions - Part 2
    16:37
  • Inline - if else and List Comprehensions
    03:35
  • Installing xlrd and XlsxWriter​ to Read/Write to Excel Files
    00:19
  • Wrting to Excel Files
    Preview10:02
  • Reading from Excel Files
    05:09
  • Python Editor & Other Software
    01:07
  • Exercise #1: YOU: Web Scraping Expert
    00:18

  • How Websites are Hosted
    01:32
  • HTML Revision
    Preview02:22

  • BeautifulSoup Solved Exercise
    01:38
  • Simple Scraper
    1 question

  • Installing Required Python Packages
    03:19

  • Requests Get Method
    08:50
  • User Agent
    06:55
  • Installing fake_useragent Package
    00:11

  • Preview04:44
  • Web Scraping with Beautiful Soup - Overview P.2
    02:56
  • Accessing Tags
    07:08
  • Navigable Strings
    03:30

  • Navigating through Tag Names
    03:33
  • Contents and Children Methods
    07:02
  • Descendants Method
    08:19

  • Parent Method
    07:02
  • Parents Method
    04:40

  • next_sibling
    04:46
  • previous_sibling
    03:52
  • next_siblings & previous_siblings
    04:38

Requirements

  • Some prior programming experience in Python (e.g. Data Structures and OOP) will help. The course includes a full Python refresher section.
  • Complete beginners may wish to take a beginner Python course first, and then transition to this course afterwards.
  • This course adopts a step-by-step approach and requires you to open a Python editor, download available *.py code files, and start applying the provided examples and exercises.
  • Python 3: Codes of this course are tested on Python 3. It is up to you to adapt them if you want to run them in Python 2.

Description

Web Scraping (also termed Screen Scraping, Web Data Extraction, Web Harvesting, etc.) is a technique for extracting large amounts of data from websites  and save the the extracted data to a local file or to a database.

In this course, you will learn how to perform web scraping using Python 3 and the Beautiful Soup, a free open-source library written in Python for parsing HTML.

We will use lxml, which is an extensive library for parsing XML and HTML documents very quickly; it can even handle messed up tags. We will also be using the Requests module instead of the already built-in urllib2 module due to improvements in speed and readability.

Finally, we will use Selenium alongside Beautiful Soup to crawl AJAX & JavaScript driven pages.

The course cover the following topics: accessing web pages programmatically; scraping web pages to extract the required data using Beautiful Soup to parse web pages; interacting with web pages to do different things with them programmatically; and using Selenium for web scraping and when we need it.

By the end of this course, you will be able to understand how websites and servers function, diverse data extraction techniques, and methods of handling and organizing data.

This Web Scraping course covers the following topics:

  • Review of data structures (Lists, Dictionaries, Tuples, File Handling)
  • How websites are hosted on servers
  • Calls to the server (GET, POST methods)
  • Review of HTML and CSS
  • Requests Module and BeautifulSoup Module overview
  • Parsing HTML using BeautifulSoup
  • Filtering elements using BeautifulSoup and navigating the Parse Tree
  • JavaScript and AJAX overview
  • Selenium and the need for it
  • Selecting elements using Selenium 
  • CSS selectors 
  • XPath selectors 
  • Navigating pages using Selenium 
  • Practical Projects



Who this course is for:

  • Those who want to learn how to use Python for web scraping and data extraction.

Instructors

GoTrained Academy
eLearning Professionals
GoTrained Academy
  • 4.3 Instructor Rating
  • 5,322 Reviews
  • 60,750 Students
  • 9 Courses

GoTrained is an e-learning academy aiming at creating useful content in different languages and it concentrates on technology and management.

We adopt a special approach for selecting content we provide; we mainly focus on skills that are frequently requested by clients and jobs while there are only few videos that cover them. We also try to build video series to cover not only the basics, but also the advanced areas.

Waqar Ahmed
Waqar Ahmed
  • 4.4 Instructor Rating
  • 981 Reviews
  • 6,446 Students
  • 2 Courses

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.