Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Software Development Tools No-Code Development
Business
Entrepreneurship Communication Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certifications Network & Security Hardware Operating Systems & Servers Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Paid Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement & Gardening Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition & Diet Yoga Mental Health Martial Arts & Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Learning Teacher Training Test Prep Other Teaching & Academics
Web Development JavaScript React Angular CSS Node.Js PHP HTML5 Vue JS
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Amazon AWS Cisco CCNA CompTIA Security+ Microsoft AZ-900
Microsoft Power BI SQL Tableau Data Modeling Business Analysis Business Intelligence MySQL Qlik Sense Data Analysis
Unity Unreal Engine Game Development Fundamentals C# 3D Game Development C++ Unreal Engine Blueprints 2D Game Development Mobile Game Development
Google Flutter iOS Development Android Development Swift React Native Dart (programming language) Kotlin Mobile App Development SwiftUI
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting Canva InDesign Character Design Procreate Digital Illustration App
Life Coach Training Neuro-Linguistic Programming Personal Development Personal Transformation Life Purpose Mindfulness Sound Therapy Meditation Coaching
Business Fundamentals Entrepreneurship Fundamentals Freelancing Business Strategy Startup Business Plan Online Business Blogging Leadership
Digital Marketing Social Media Marketing Marketing Strategy Internet Marketing Google Analytics Copywriting Email Marketing Startup YouTube Marketing

DevelopmentProgramming LanguagesPython

Web Scraping with Python: BeautifulSoup, Requests & Selenium

Web Scraping and Crawling with Python: Beautiful Soup, Requests & Selenium
Rating: 4.0 out of 54.0 (876 ratings)
6,120 students
Created by GoTrained Academy, Waqar Ahmed
Last updated 12/2018
English
English, Italian [Auto]

What you'll learn

  • Python Refresher: Review of Data Structures, Conditionals, File Handling
  • How Websites are Hosted on Servers; Basic Calls to Server (GET, POST Methods)
  • Web Scraping with Python Beautiful Soup and Requests
  • Using Selenium to handle JavaScript and AJAX
  • Diverse Web Scraping Exercises
  • Source codes (*.py files) for all Exercises can be downloaded
  • Q&A board to send your questions and get them answered quickly

Requirements

  • Some prior programming experience in Python (e.g. Data Structures and OOP) will help. The course includes a full Python refresher section.
  • Complete beginners may wish to take a beginner Python course first, and then transition to this course afterwards.
  • This course adopts a step-by-step approach and requires you to open a Python editor, download available *.py code files, and start applying the provided examples and exercises.
  • Python 3: Codes of this course are tested on Python 3. It is up to you to adapt them if you want to run them in Python 2.

Description

Web Scraping (also termed Screen Scraping, Web Data Extraction, Web Harvesting, etc.) is a technique for extracting large amounts of data from websites  and save the the extracted data to a local file or to a database.

In this course, you will learn how to perform web scraping using Python 3 and the Beautiful Soup, a free open-source library written in Python for parsing HTML.

We will use lxml, which is an extensive library for parsing XML and HTML documents very quickly; it can even handle messed up tags. We will also be using the Requests module instead of the already built-in urllib2 module due to improvements in speed and readability.

Finally, we will use Selenium alongside Beautiful Soup to crawl AJAX & JavaScript driven pages.

The course cover the following topics: accessing web pages programmatically; scraping web pages to extract the required data using Beautiful Soup to parse web pages; interacting with web pages to do different things with them programmatically; and using Selenium for web scraping and when we need it.

By the end of this course, you will be able to understand how websites and servers function, diverse data extraction techniques, and methods of handling and organizing data.

This Web Scraping course covers the following topics:

  • Review of data structures (Lists, Dictionaries, Tuples, File Handling)
  • How websites are hosted on servers
  • Calls to the server (GET, POST methods)
  • Review of HTML and CSS
  • Requests Module and BeautifulSoup Module overview
  • Parsing HTML using BeautifulSoup
  • Filtering elements using BeautifulSoup and navigating the Parse Tree
  • JavaScript and AJAX overview
  • Selenium and the need for it
  • Selecting elements using Selenium 
  • CSS selectors 
  • XPath selectors 
  • Navigating pages using Selenium 
  • Practical Projects



Who this course is for:

  • Those who want to learn how to use Python for web scraping and data extraction.

Instructors

GoTrained Academy
eLearning Professionals
GoTrained Academy
  • 4.0 Instructor Rating
  • 6,121 Reviews
  • 71,744 Students
  • 9 Courses

GoTrained is an e-learning academy aiming at creating useful content in different languages and it concentrates on technology and management.

We adopt a special approach for selecting content we provide; we mainly focus on skills that are frequently requested by clients and jobs while there are only few videos that cover them. We also try to build video series to cover not only the basics, but also the advanced areas.

Waqar Ahmed
Waqar Ahmed
  • 3.8 Instructor Rating
  • 1,035 Reviews
  • 6,873 Students
  • 2 Courses

Top companies choose Udemy Business to build in-demand career skills.
NasdaqVolkswagenBoxNetAppEventbrite
  • Udemy Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Investors
  • Impressum Kontakt
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Accessibility statement
Udemy
© 2022 Udemy, Inc.