
Discover AWS Glue, an ETL service that discovers data, catalogs schemas with Glue Catalog, and runs Python or Spark transforms to load into Redshift or S3, with Athena and QuickSight.
Explore aws data engineering with rds and aurora, a fully managed, scalable relational database in vpc, plus redshift for data warehousing and s3 querying with spectrum.
Explains incremental data loading by comparing a csb file with the database to extract new records, save them to a 063 file in s3, and load them into the table.
A hands on course that covers majority of the typical data engineering / ETL scenarios.
In this course you will learn:
Different services and concepts of AWS data engineering
Creating serverless data lake using S3, Glue and Athena
Ingesting data using Rest Api
Ingesting data using Sftp server
Ingesting data into Database (AWS RDS - Postgre SQL)
Incremental data loading
Prerequisites:
An active AWS account
Python / SQL knowledge