Building Web Scraping Bot With Python - Absolute Beginners

Learn how to scrape data by understanding web page structure, inspecting source code and tags, and using Python tools like Requests and BeautifulSoup to extract titles and text.

Explore web scraping with Python, using Beautiful Soup to extract stock prices from finance.yahoo.com, compare Selenium and other libraries, and follow ethical guidelines by checking robots.txt.

Install the requests and BeautifulSoup libraries, import them, fetch a page with requests, and add a user-agent header to mimic a browser for parsing with BeautifulSoup.

Learn how to use BeautifulSoup in Python to parse web pages with a parser and build a web scraping bot, extracting titles and paragraphs with find, find_all, and get_text.

Build a Python web scraping bot that extracts required data, like the page heading and current stock price, by targeting specific HTML tags and printing the results.

Extract the current price from a web page by inspecting elements, filtering a fixed attribute, and selecting the first span inside the target class, handling dynamic attributes.

Extract table content from a web page by targeting the div and class, then parse headings and values like previous close, open, bid, and ask, looping to capture all items.

Learn to extract table content using beautiful soup, looping over indices to capture header and value pairs, and refine selectors with td and span.

Learn to extract stock results by looping over a stock list with a for loop, fetch each URL from Yahoo Finance, and apply a five-second delay between requests.

Learn how to write scraped data to a csv file using a writer, create a heading row, and store stock data and current price values from each row.

Build a process to store stock data in a csv file by creating a list of stock titles and prices, looping to populate it, then writing and closing the file.

learn how to send text emails (with and without attachments) using smtplib in Python, connect securely to Gmail, configure app passwords, and compose messages with subject and body for testing.

Learn to send a text email using Python’s email module by building a multipart message with from, to, and subject headers, a plain text body, and optional attachments.

Learn to send an attachment through the Python email module by reading the file in binary mode, encoding it with base64, setting content disposition, and attaching the payload with message.attach.

Integrate a mail system with your web scraper by turning the script into a callable function, passing a file name variable, and updating the email subject and body.

Create daily, date-based file names by importing date, converting it to a string, and concatenating it into the file name and email subject.

You unlocked a new set of web scraping skills by following this course. Leave a review to help motivate updates and more lectures in the future.