Web Scraping for Data Science - Python & Selenium - Basics

Master the basics of XPath, including syntax and building XPath expressions for web scraping. Compare relative and absolute XPath, single and double slashes, and understand parents and siblings logic.

Discover why XPath locators matter in web scraping. Learn to compare ID, class, and link text options and craft XPath expressions to target elements or lists in the DOM.

Apply XPath syntax to craft precise expressions for HTML elements, using //, tag, and [@attr='value'] to locate a unique element for web scraping.

Explain the difference between absolute XPath and relative XPath, showing root start paths versus paths that start from an element, and how double slashes help target the top seller text.

Learn the difference between single slash and double slash in XPath expressions, and how to target immediate vs. non-immediate child elements with relative and absolute paths.

Master XPath techniques by starting from a child element and navigating to its parent or to preceding and following siblings, using index-based selection and the parent, preceding-sibling, and following-sibling axes.

Set up selenium for web scraping by installing selenium and Jupyter notebook, downloading and configuring the Chrome driver, and outlining driver paths for Windows.

Discover selenium basics, its use cases for testing automation and web scraping, after setting up chrome driver and jupiter notebook, with locator strategies like XPath, class name, and id.

Install and import Selenium packages, including WebDriver, Keys, By, and Options, to initialize the Chrome driver, store URLs in variables, and open them for web scraping practice.

Explore the XPath locator to extract a product price, compare absolute and relative paths, and retrieve single and multiple price elements with Selenium.

Use the class name locator in Selenium with Python to collect 40 product names, print each name, and verify the list length and text content from the page.

Learn how to locate a web element by its id using Selenium, click the search area, and input text with send keys, including basic waits.

Create a name locator for the search button using the name attribute 'submit search', assign it to a variable, and click to submit the search, demonstrating a name locator.

Learn how to use link text and partial link text locators in Selenium to find and click contact us button using driver.find_element by link text and by partial link text.

Explore alternative XPath syntax in selenium, illustrating two locator options: XPath locator and ID locator. Build and test XPath expressions with driver.find_elements, enhancing element selection in Python and Selenium.

Explore Selenium in action to automate web interactions for data scraping, including launching the driver, navigating pages, performing searches, clicking links, and closing the browser, with an optional headless mode.

Explore how to run Selenium scripts in headless mode using Chrome options, verify actions with print statements, and learn the basics of headless web automation for data science.

Set up the driver, import the necessary packages, handle cookies, build locators, create a data frame, and store the scraped data in an Excel file.

Discover how to build and validate locators with XPath for web scraping, extracting names, telephone numbers, addresses, expertise, stars, and writing code with Selenium in Python.

Demonstrates building a pandas data frame from scraped results by creating lists, looping to append data, and converting a dictionary to a data frame for export to Excel after cleaning.

Learn to clean messy scraped data by removing the star text from the stars column using a data frame and a lambda replace, then save to Excel.

Set up and initialize Selenium to scrape a pokemon database table and export the data to an Excel file, while handling the privacy popup.

Store scraped data by building dictionaries, collecting them in a list, and converting to a pandas data frame for cleaning before exporting to Excel.

Learn how to clean scraped data in Python by replacing newline and backslash characters in a Pokemon data frame, using pandas apply and lambda, to prepare for an Excel export.

Save the cleaned data frame to an Excel file named pokemon_database_cleaned.xlsx without an index, preserving the scraped Pokémon data and allowing quick formatting and review.