
Install Jupyter Notebook, requests, pandas, and SQLAlchemy for Python web scraping. Open notebooks from the command prompt and connect data frames to a PostgreSQL database.
Install PostgreSQL on Windows, set up PGAdmin, and establish a database connection. Then learn to output a data frame into the database and practice SQL queries.
Learn how to scrape table data with Python and Selenium in a Jupyter notebook, and save results to CSV, JSON, Excel, and PostgreSQL using SQLAlchemy.
Learn Selenium based table scraping in Python to extract country data (populations 2021 and 2020, growth rate, area, density) and organize results into a pandas data frame.
Export scraped table data to Excel, JSON, and a PostgreSQL database using SQLAlchemy, from the population dataframe. Learn encoding, separators, and index options while saving the data.
Scrape table data using Beautiful Soup and Python, create a data frame, and export results to CSV, Excel, JSON, and a SQL database.
Use beautiful soup to extract table data into per-field lists (name, city, completion year, height, floors, material, function), build a pandas data frame, and export to csv, excel, and json.
Export scraped table data to csv, excel, json, and postgresql using dataframe methods and sqlalchemy. Learn to set separator, utf-8 encoding, and disable index for clean exports.
Explore api-based table data extraction from the Ontario Hockey League site using Python requests and pandas to pull players’ name, age, games played, goals, points, penalty minutes, and home country.
Extend table scraping to all 100 players by collecting name, height, games played, goals, points, penalty minutes, and country, then build a pandas DataFrame for csv, json, and postgresql.
In this Project we will scrape data from Tables, using three different approaches : Selenium - Beautiful Soup and API Scraping.
Especially for people who work with data it is important to be able to create own datasets. Often we rely on datasets from someone else. This course should show all data enthusiasts how to scrape and store data in Excel Files, CSV, JSON and in the PostgreSQL database.
The requirement for this course is basic knowledge of Python Programming. Since we will not cover very difficult Python topics you do not have to be a professional. The most important characteristic is that you are curious about Web Scraping and Data Mining. You should be ready to invest time in gaining the knowledge which is taught in this course.
After this course you will have the knowledge and the experience to scrape your own data and create your own dataset. With the help of the course resources you will always have documents you can refer to. If you have a question or if a concept just does not make sense to you, you can ask your questions anytime inside the Q&A - Forum. Either the instructor or other students will answer your question. Thanks to the community you will never have the feeling to learn alone by yourself.
What you’ll learn
Web Scraping
Output Data in Excel
Output Data in CSV
Output Data in JSON
Output Data in PostgreSQL
Data Extraction
Web Scraping for Data Science
Data Mining
Data Scraping
Are there any course requirements or prerequisites?
Basic understanding of Python Programming
Who this course is for:
Everybody who is interested in Web Scraping, for example Data Scientists
Professionals who want to create their own dataset without being dependent on some else
Are there any course requirements or prerequisites?
basic understanding of Python programming
Who this course is for:
Data Enthusiasts who want to create their own datasets
Disclaimer : I teach web scraping as a tutor for educational purposes. That's it.
The first rule of scraping the web is: do not harm a certain website. The second rule of web crawling is: do NOT harm a certain website.