Automating ETL - Complete Code Challenges and ETL Projects
4.1 (42 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
250 students enrolled

Automating ETL - Complete Code Challenges and ETL Projects

From beginner to advanced after challenges and projects. Free seminar access for lifetime students.
4.1 (42 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
250 students enrolled
Created by Tim Smith
Last updated 7/2020
English [Auto]
Price: $89.99
30-Day Money-Back Guarantee
This course includes
  • 13 hours on-demand video
  • 4 articles
  • 15 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Introduce ETL effectively
  • Learn and understand extract
  • Learn and understand transform
  • Learn and understand load
  • Learn the theory behind automation
  • Learn how to automate the ETL process
  • Learn advanced techniques in extract, transform and load
  • See case studies with applied ETL uses for freelancers or business owners
Course content
Expand all 64 lectures 12:57:44
+ Introduction
2 lectures 14:26

Students: this is an about video for people looking at this course and is not required for course completion.

Preview 11:24
+ Automating ETL Processes - Extract
14 lectures 02:20:45
Extract Introduction
Extract (Part One)
Extract (Part Two)
Extract (Part Three)
Extract (Part Four)
Extract (Part Five)
Extract (Part Six)
Extract (Closing Thoughts)
SSIS & SQL Server Data Tools Overview
SSIS Project: Aggregate Regional Weather Data

This is the extract code challenge 1 that will be required to complete before the transform code challenge 1.  In addition, you are required complete all code challenges before the optional practice interview.

Extract Code Challenge 1
Extract Code Challenge 2
Our Course Project (Extract Homework)
3 Additional Course Projects
+ Automating ETL Processes - Load
6 lectures 01:34:25
Load (Part One)
Load (Part Two)
Load (Part Three)

As a note, you will want to complete the Extract Code Challenge and Transform Code Challenge 1 before this assignment.

Load Code Challenge 1

As a note, you will want to complete the Extract Code Challenge and Transform Code Challenge 2 before this assignment.

Load Code Challenge 2
Our Course Project (Load Homework)
+ Automating ETL Processes - Transform
11 lectures 02:43:10
Transform (Part One)
Transform (Part Two)
Transform (Part Three)
Transform: (Part Four)
Transform (Part Five)
Transform (Part Six)
Our Course Project (Transform Homework)

You are required complete all code challenges before the optional practice interview.

Transform Code Challenge 1

You are required complete all code challenges before the optional practice interview.

Transform Code Challenge 2
Our Course Project (Transform Homework Part II)
Transform Coding Live Practice (Optional)
+ ETL and Performance
3 lectures 42:40
Performance Overview
Performance Part 1
Performance Part 2
+ ETL In the Cloud
1 lecture 16:01

See the resources section for materials listed.

ETL In the Cloud Part 1
+ Live Practice and Examples
4 lectures 01:03:55

I got a call about an opportunity that someone described as cheap and that I should take advantage of it immediately.  I didn't know if it was or not, so using the tools in this course, I spent a few minutes importing data and doing a quick analysis.  Was it a good opportunity?

Good Deal Or Not? Practice

While I hope my concerns never come true, ask yourself based on the data I present - is this as good as the real thing?  People may be engaging in a mistake without realizing it.  With these ETL tools, you will speed up your ability to do research like this.

Is This As Good As The Real Thing?

This video shows excellent analysis by my friend, but with the wrong assumption, a high cost.  I hope all of you can avoid this.  I've experienced this as well and it was a hard lesson learned!  As a bonus, I've attached the CSV in downloadable materials in this lecture.

Be Extremely Careful - Data Example

What are steps that we can take to get data or assess data when we can't automatically extract data?  Looking at an example of assessing a subjective value (that can't be measured or wasn't measure at the time), we go through seven steps.

How To Handle A Major Extract Challenge?
+ Optional Interview Practice and Reference Requests
1 lecture 05:32
Preparing For the 30 Minute Practice Interview
+ The Master Class Series
13 lectures 02:52:00
Master Class Series Introduction
The Master Logic of Extract
Data Gathering
Handling Dirty Excel Delimiters
Another Excel Extract Method
Handling Wrapped Exceptions
Handling Useless Characters
Excel Non-Data Sheets
Automate Reporting
Automating Email Loading
ETL Architecture
Your ETL Briefcase
Thinking About ETL In Big Data Environments
+ Optional Exercises
1 lecture 00:09

The following series is an optional exercise and is not a part of the 5 ETL projects.  Because some environments may require we use other languages, we look at what we can do if we need to swap languages.  In this file, we look at situations where we may be required to swap code or functions into another language.  We also look at a character challenge that's a popular question people ask.

Python Swapping
  • Mandatory: Computer with Microsoft SQL Server Express (or above) 2008R2 (or above)
  • Mandatory: PowerShell ISE 3.0 (or above)
  • Optional: Visual Studio 2010 (or above)
  • Optional: MongoDB
  • Optional: SQL Server Data Tools (SSIS)

This course covers a guide to automating ETL through automation techniques, intended for business owners, freelancers or employees seeking to jump ahead of the competition and seeking to minimize data disasters. With the demand for ETL developers increasing (per the Bureau of Labor Statistics and the opportunity to become a data provider (for business owners), I highlight methods and architecture to minimize the work that must be done for ETL, so that you can spend more time working on enhancing skills. 

I offer a ton of free content on my YouTube channel (SQL In Six Minutes), and suggest students to become familiar with the basics of SQL Server and MongoDB, covered at the below, free playlists: 

  • SQL In Six Minutes (YouTube)

  • The MongoDB 103 (YouTube)

In addition, the only coupon for this course that I offer is available in the trailer for my YouTube channel.  Since this is the most popular question I get asked, see the trailer for the YouTube channel to get it.

This course covers: 

  • Understanding the extract process and constructing architecture to automate it.

  • Understanding the load process, when change is necessary with loading, and how to automate it.

  • Understanding different issues we may face in transform, how to minimize re-facing them with re-usable solutions.

  • Understanding advanced techniques in ETL, the theory behind these, and applications with them.

  • Understanding the basic logic, functions, operators and advanced use of PowerShell so beginners and mid-level developers can advance quickly with it.

  • Completing a large assignment that will have you import over 200K of records.

  • Reviewing case studies of applied ETL techniques, especially useful for freelancers adding credibility to their business.

  • Course features extra ETL content with MongoDB and SSIS/SQL Server Data Tools - though neither of these are required for this course.

To provide maximum learning for students wanting to master ETL skills, whether for business opportunity or employment, the course provides the following material: 

  • New content posted every year since this course began which offer students a look into the new challenges faced by ETL developers.

  • Q&A responses within 48 hours for students who need assistance along with ongoing discussions that students may find helpful.

  • Tri-annual bonus material for long-term and successful students in this course based on several metrics provided by Udemy.  Technology always changes which is why this course continues to be updated and built with rewards for long time students.

  • Focused video lectures with demonstrations of ETL processes in action, the code/script used to make the process occur, and no distracting facial or background movements.

  • Dynamic syllabus with extra content, such as links to articles and videos to enhance and increase your learning, as well as links to specific recruiters in desperate need for ETL developers (students wanting to become employees) and ways to uncover possible data providing opportunities for business owners.

  • Scripts/code that are used in the video lectures, as well as additional tools that students can learn as they construct their own ETL architecture.

  • Course material that we use in the course, such as files we import during the series.

  • Course projects that you complete as well as your own project to develop.

  • Feedback session for students who meet the requirements where you practice answering questions and presenting your own project.

Understanding, manipulating and transforming data will only become more important in the future.  Those who can master this through automation will have an advantage over all their competition because when we see the results of our measurement, only then can we make the appropriate changes; this course shows you how to automate those steps. 

Companies seeking ETL talent.  If you are a company seeking junior to senior ETL developers, you can get in touch with me, as I have metrics on students who completed exceptional work in this course and have expressed interest in working with a firm.  In addition, with some of these students, I've had feedback sessions where I've assessed their skills, helping you identify who may be solid candidates.  You can find my contact information on my YouTube about page, or on my LinkedIn profile.

Who this course is for:
  • Businesses requiring ETL developers
  • Students interested in ETL positions
  • Employees working in ETL positions
  • Employees interested in ETL positions
  • Individual business owners who want to perform ETL on their own
  • Employers needing ETL developers
  • Recruiters wanting to understand ETL positions