Data Engineering with Snowflake and AWS
What you'll learn
- Tasks of a Data Engineer in Snowflake
- How Snowflake platform can support engineers
- Some custom SQL Snowflake code
- Extraction, Transformation and Data Loading
- What you need in AWS to integrate Snowflake
- ETL
- Create Amazon S3, IAM Role and Policies, SNS topics
Requirements
- Familiarity with SQL is recommended but not mandatory
- Familiarity with AWS is recommended but not mandatory
Description
Snowflake course for data engineers
This comprehensive Snowflake course is designed for data engineers who want to improve their ability to efficiently and scalably manage data in the cloud. With a hands-on focus, participants will be guided from the basics to advanced concepts of the Snowflake platform, which provides a modern and fully managed data warehouse architecture.
Benefits of using Snowflake for data engineering:
Elastic scalability: one of the key benefits of Snowflake is its cloud data storage architecture, which allows for elastic scalability. This means that data engineers can easily scale resources on demand to efficiently handle variable workloads and ensure consistent performance regardless of data volume.
Simplified data sharing: Snowflake offers a unique approach to sharing data across departments and teams. Using the concept of secure and controlled data sharing, data engineers can create a single data source that promotes efficient collaboration and consistent data analysis across the organisation.
Seamless integration with analytics tools: Snowflake is designed to integrate seamlessly with a variety of data analytics tools, allowing data engineers to create complete ecosystems for advanced data analysis. Compatibility with standard SQL makes it easy to migrate to the platform, while interoperability with popular tools such as Tableau and Power BI expands options for data visualisation and exploration.
In this course we deal with:
Snowflake basics
Platform architecture
Virtual warehouses - the clusters
Working with semi-structured data
Integrating Snowflake with AWS
Using Stages, Storage Integration, and Snowpipes
Using AWS S3, SQS, IAM
Automatic ingestion of data in near real time
Who this course is for:
- Data Engineers
- Data Analysts
- Database Administrators
- Analytics Engineers
- Cloud Engineers
- Software Engineers
- Database Developers
- Python Developers
- Data Managers
- Data Leaders
Instructor
I'm self taught Senior Data Engineer and content creator. Migrated from a machine operator at my 30's to the Data IT Industry. Can help early professionals to drive their path to become data professionals as well as give some great advices for those who wish to live abroad and achieve a sponsorship visa.
My current stack:
Data Integration / Processing -> Databricks | Dataflow | AWS Lambdas | Datafusion | DataFactory
Automation -> Power Platform | Power Automate | Power Apps
Databases -> Snowflake | Big Query | SQL Server
Data Transformation -> DBT
Versioning / Repository -> Git | Azure DevOps
Programming -> SQL | Python | PySpark
Cloud Providers -> Azure | GCP | AWS
Task / Data Orchestration -> Airflow
BI -> Power BI | Qlik Sense
CI / CD -> Git Lab CI
Containers -> Docker