Essential Data Science: Database and ETL With Python

Explore folder processing and file management with python 3, and practice reading json and xml files from websites, while handling security considerations and object store protocols for accessing resources.

Learn basic Python I/O for data science: list and filter files and folders, navigate directories, and perform create, copy, rename, move, and delete operations on files and directories.

Read and process text files using pandas by specifying delimiters, creating dataframes, and writing results back to text files without indexes, including verifying columns like id and gender.

Explore reading csv files with pandas by specifying delimiters such as comma or semicolon. Verify results by testing with a sample file and setting index=false.

install openpyxl and read Excel files in pandas, then read data with read_excel, optionally rename columns, and export to a new Excel file using to_excel, verifying content matches.

Learn to work with json data in python by reading, creating, and verifying json records with pandas, updating fields such as id, name, email, gender, and orientation.

Demonstrate reading an XML file using pandas, traverse root to employee elements, build a dataframe with email, gender id, and name, and render a formatted XML string for display.

Learn how to read a file from an http website using pandas, handling basic authentication via requests, streaming the response, and loading from the stream.

Learn to read files from server-based S3 using Python with pandas and boto3, including configuring access keys, endpoints, and reading object content from S3 buckets.

Learn basic database concepts and data manipulation, including implementing steps to delete data, and understand its use in a data science workflow.

Learn to create a SQLite database, define tables such as employee and product, perform insert, update, delete, and select operations, and use parameters in queries with pandas.

connect to a mysql database with python, create and manage a product table, perform insert, update, delete, and select operations, and load data with pandas.

Install python and sql server locally on ubuntu, create a products table with name, price, quantity, and quality, and perform insert, select, update, and delete operations using pandas.

Connect to PostgreSQL from Python, create and manage tables with insert, select, update, delete operations and commit, and return inserted IDs, while using pandas to load and verify data.

Explore how to implement create, read, update, and delete operations in a database to manage data effectively.

Explore MongoDB basics: create and connect to a database, manage collections, insert, read, update, and delete documents, and drop collections to manage ETL workflows in Python.

Learn how to use Redis with Python: connect to a local server, set, get, update, and delete keys, scan keys, and transfer data between a data frame and Redis.

Learn to set up Apache Cassandra locally, create keyspaces and tables, perform insert, update, and delete operations, and load data with pandas for Python-based ETL workflows.

presents case studies on multiple file datasources, deduplicating data, and loading into database tables with pandas, including reading an email from the Meltzer file as a datasource.

Execute an etl workflow that reads multiple files from a folder using pandas, removes duplicates, and loads the cleaned data into a MySQL database.

This lecture demonstrates extracting data from multiple files and a database, combining content, removing duplicates, and loading the cleaned time frame into a target database using a python-based etl workflow.

Read email from imap, extract sender, subject, and body, reconstruct into a dataframe, and load into a database table as part of a Python etl workflow.