
Have a clear view of what valuable capabilities you will get after this course
Understand how ETL and testing process works
Get to know the tool list that is needed for ETL
Learn more from live documents shared by Microsoft
Learn the prerequisite for this course
Be able to set up Azure databricks service on your own after this lecture
Be able to set up Azure cosmos DB service on your own after this lecture
Be able to connect Azure databricks service to cosmos DB on your own after this lecture
You can load data into cosmos DB by yourself with the tool shared in this lecture
You can understand the project background and where business requirement comes from
You are able to understand what legacy data we have before commencing the open banking project.
You can understand how to convert business requirement to technical requirement
You should be able to understand why and how to design the solution for the open banking project
You should have a clear view about the targets we are going to achieve for this project and how to create test strategy to achieve the goals.
Static testing is part of shift-left testing approach. You will know what to cover in static testing.
You can extract data from source system on your own after this lecture
Know why and how to add a prefix for data frames.
You will be able to start transforming your own data after this lecture
You can do a proper transformation based on business rules after this lecture
You can understand the reason why we need a validation hash and how to add it
You should understand why we have to sort records and how to sort them after this lecture
You will make your code neat and easy to maintain after code refactory
You will have the capabilities of grabbing the expected new recoreds to be loaded into cosmos DB
You will have the capabilities of grabbing the expected modified records to be loaded into cosmos DB
You will have the capabilities of grabbing the expected records to be deleted from cosmos DB
You are able to resolve stability issues during data ingestion process
You are able to resolve performance issues during data ingestion process
You will see the physical paths for the tables saved on delta lake and understand how to use the paths
You can delete any records in cosmos after this lecture
You will have the capability of ingesting data into cosmos
You will know how to resolve performance issue during data ingestion process via maintaining a cosmos clone table in delta lake
You are able to resolve reliability issues during data ingestion process
Are you thinking of getting higher income for a long time with a short training period? Then, you come to the right place. As you know AI and big data are the driving forces behind Industry 4.0. Data engineer is high-paid role. As the shortage of data engineers continues and the demand for skilled data engineer is soaring. Switching to data engineer is one of the best options for your career, which can get higher pay, long-term contracts and less pressure. By joining this course, you will know how the front-line data engineers execute ETL with real open banking project from the scratch, get the ready-to-use, practical and powerful ETL framework and the ISTQB testing strategy. So, it will be highly possible to pass your job interview because you really have the similar capabilities with data engineers who earn $200, 000+ yearly and be able to resolve complex ETL issues on your own after this course.
If you are already a data engineer, this course will sharpen your ETL skills. This course can free yourself via saving huge amount of computing time and cost during ETL process. More importantly, you will get the capabilities of resolving complex issues in terms of performance, stability and reliability. You will enjoy more free time and leave the hard work to the ETL framework and scripts.
With 6 sections and about 5 hours of high-quality content, you will learn the following topics in depth:
Section #1: Values We Can Bring to You
You will understand what valuable capabilities you can take away after this course and extra free live resources you can keep learning.
Section #2: Azure Databricks and Cosmos DB Setup
You are able to set up Azure Databricks, Cosmos DB and connect them on your own.
Section #3: ETL Project Initialization
Get real commercial open banking project experience from scratch including grabbing business requirement, Coverting it to technical requirement, designing the solution and implementing it.
Section #4: ETL and Testing Business Goal and Test Strategy
Identify business goals we are going to achieve and adopt right test strategies and plan static testing and shift-left testing approach.
Section #5: Data Extraction and Transformation
Implement data extraction and transformation functionalities and create a smart mechanism to detect if data is modified with lower cost in terms of time and efficiency.
Section #6: Stable and Speedy Data Ingestion
Resolve complex ETL issues in terms of performance, stability and reliability, etc. as an expert and get the powerful framework and scripts and customize them and apply to your own projects.
Last but not least:
Enjoy this course and don't forget to check out my other courses if you want to learn how to test the project with ISTQB recognized automation testing framework:
2023 Databricks ETL Testing Services Complete Guide