Python Webscraping For Information Retrieval and Analytics

Mount your Google Drive in a Colab notebook to access datasets, authorize the drive, and import files from your drive for data analysis in Python webscraping and information retrieval.

Learn to open a Jupyter notebook in Google Colab from Google Drive or a local upload, import libraries, and access drive files for use in the notebook.

Access data from Google Drive in a Python notebook by mounting drive, navigating folders, selecting a file (cancer tumor), and reading it with read_csv for analytics.

Upload data from your local drive, drag and drop files, and read them into your Python session using file upload and IO operations.

Install and verify PySpark, create a Spark session, and test a dataframe to confirm installation; repeat Java, Spark, Hadoop, and path setup if Colab issues occur.

Explore how web scraping pulls data from HTML pages to create structured datasets for insights from Wikipedia, Amazon, and property sites, while noting legal and ethical considerations.

Explore how a web page is structured by rummaging through example pages, view page source, and understand HTML, doctype, head, and images.

Explore HTML fundamentals, from the doctype declaration to tags and attributes. Learn how HTML documents form a tree with head and body, enabling scraping of links, headings, and tables.

Learn how to access different html components using python or browser inspect, identify a table and its data cells, and reference them through parent tags for web scraping.

Explore extracting information from HTML pages with Beautiful Soup, using tag and class names to retrieve text, and learn how to install and run it in Google Colab or Jupyter.

Learn to perform simple webscraping by reading HTML into BeautifulSoup with Python, parse a page such as a Wikipedia entry, and access tables and links.

Learn another way to read HTML pages with requests and BeautifulSoup. Parse HTML and extract tabular data from wiki tables and other pages, illustrated with Wikipedia and Macrotrends Tesla revenue.

Learn to extract specific table data from html pages with Beautiful Soup. Focus on table structures, including table, tr, and td, to capture crop-pollinator details.

Learn to extract multiple tables with requests and BeautifulSoup, count them, and access each via zero-based indexing, retrieving data from Central Asia and East Asia tables.

Discover a quicker way to extract tabular data from HTML pages using a streamlined approach with requests, BeautifulSoup, and pandas, parsing multiple tables into a list.

Learn how to extract and print table names from multiple html tables using beautiful soup, enabling quick access by indices for information retrieval and analytics.

Learn to extract multiple html tables from a Wikipedia page about World Heritage sites in the Philippines using pandas read_html, then select by index to build data frames.

Learn to scrape a simple non Wikipedia page using requests and BeautifulSoup, extracting the first IPL table from 2020–2021 and checking for a successful 200 response.

Extract a wiki table of Canadian postal codes with BeautifulSoup, capturing postal code, neighborhood, and related fields, then format them into a tidy Toronto data frame.

Scrape IPO listings from a site using requests, BeautifulSoup, and pandas into a dataframe. Rename and organize columns like company symbol, industry, offer date, shares, and offer price for analysis.

Learn to clean IPO data by stripping dollar and percentage signs, cast to float64, and describe quantitative metrics in a data frame for analytics.

Check terms and conditions and permissions before scraping, then set a user agent header to access non Wikipedia pages.

Learn to scrape a London Airbnb listing page using requests and BeautifulSoup, extracting listing names, prices, and descriptions from item list elements.

Learn to scrape Amazon best sellers pages by targeting item containers and class attributes, and use selectors to extract book titles, prices, ratings, reviews, and product links.

Scrape amazon bestsellers into a pandas dataframe by fetching five pages with requests and BeautifulSoup, extracting book name, author, rating, and price.

Scrape Mumbai apartment price data from a real estate site, extract locality, min and max price per square foot, average price, and price rise into a data frame for analysis.

Explore how the pandas package structures data with series and data frames, enabling labeled indexing and powerful analytics, and learn to read external data into data frames for analysis.

Drop passenger id, name, and ticket columns to clean the data. Inspect nulls with isnull and fill age and fare with gender-based means, then replace cabin and embarked values.

Learn how to transform scraped web data into a clean pandas DataFrame ready for analytics, starting with a simple Mumbai neighborhoods table and applying essential data cleaning.

Clean and extract a Mumbai locality table using web scraping and pandas. Split the location column by comma to create zone and neighborhood and yield a usable dataset.

Apply an alternate data cleaning method by parsing a web table with requests and BeautifulSoup, using pandas read_html, and splitting location into zone and neighborhood while removing missing values.

Learn how to clean a London areas data frame by renaming columns, removing brackets and references, splitting postal codes, and filtering for London rows using pandas operations.

Learn practical data cleaning in pandas by renaming columns, stripping brackets and digits, splitting postal codes, and filtering to London areas using lambda mappings and dataframe operations.

Geocode London borough names to latitude and longitude using a geocode function and a rate limiter, then map the results from locational web scraping.

Export your data frame from the London dataset to an external file in a chosen folder, mounting Google Drive in Colab and specifying the save path for later analysis.

Integrate geocoded London borough data with latitude, longitude, and socioeconomic indicators to reveal spatial and demographic patterns; prepare for fuzzy string matching with the Jaccard index.

Prepare two geocoded datasets for joining by cleaning names, renaming columns, and creating a dummy key for vanilla merges, then introduce fuzzy joining with fuzzywuzzy.

Use fuzzy wuzzy to match areas by measuring similarity with ratio, partial ratio, token sort ratio, and saturation, then merge data frames on the most similar area.

Merge datasets by geolocation using latitude and longitude to align Mumbai localities with property prices within a defined radius, such as 1 kilometer or 500 meters.

Explore theoretical principles of data visualization and select charts for categorical and discrete data, including bar and pie charts, line charts for time series, histograms, box plots, and scatterplots.

Analyze IPO data from 2020 and 2021, clean and cast it, compute sector counts, identify health care as top sector, and visualize with bar plots and pie charts.

Group IPO data by industry in a pandas data frame to evaluate sector performance, revealing technology IPOs at 49% profits and blank check companies at 2.2%.

Utilize a Mumbai property price dataset with areas and locations to bin continuous change values into discrete categories with pandas, highlighting areas with notable increases for investors.

Learn how to fetch geolocations for Singapore's MRT stations using a practical API, requests, and data frames to assemble latitudes, longitudes, and addresses for analysis.

Learn numpy, the numerical python package, and how its ndarrays enable fast, homogeneous data storage and high-level math, with import conventions like np and pd in data science.

Position the Posit cloud to deploy and share data science projects directly in your browser, using RStudio or Jupyter, with interactive apps like Shiny, Streamlit, and Dash, and GitHub integration.

Explore distributed computing frameworks that split large problems into tasks, assign them to nodes, and process in parallel, with fault tolerance and scalable resources, like Hadoop and Apache Spark.