Automating ETL - Complete Code Challenges and ETL Projects
4.1 (42 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
250 students enrolled

Automating ETL - Complete Code Challenges and ETL Projects

From beginner to advanced after challenges and projects. Free seminar access for lifetime students.
4.1 (42 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
250 students enrolled
Created by Tim Smith
Last updated 7/2020
English
English [Auto]
Price: $89.99
30-Day Money-Back Guarantee
This course includes
  • 13.5 hours on-demand video
  • 4 articles
  • 16 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Introduce ETL effectively
  • Learn and understand extract
  • Learn and understand transform
  • Learn and understand load
  • Learn the theory behind automation
  • Learn how to automate the ETL process
  • Learn advanced techniques in extract, transform and load
  • See case studies with applied ETL uses for freelancers or business owners
Course content
Expand all 66 lectures 13:20:39
+ Introduction
2 lectures 14:26

Students: this is an about video for people looking at this course and is not required for course completion.

Preview 11:24
+ Automating ETL Processes - Extract
15 lectures 02:35:33
Extract Introduction
04:19
Extract (Part One)
18:05
Extract (Part Two)
37:27
Extract (Part Three)
13:34
Extract (Part Four)
08:08
Extract (Part Five)
02:57
Extract (Part Six)
15:14
ETL Best Practices With APIs
14:48
Extract (Closing Thoughts)
03:21
SSIS & SQL Server Data Tools Overview
13:57
SSIS Project: Aggregate Regional Weather Data
07:37

This is the extract code challenge 1 that will be required to complete before the transform code challenge 1.  In addition, you are required complete all code challenges before the optional practice interview.

Extract Code Challenge 1
02:23
Extract Code Challenge 2
02:05
Our Course Project (Extract Homework)
09:36
3 Additional Course Projects
02:02
+ Automating ETL Processes - Load
6 lectures 01:34:25
Load (Part One)
20:34
Load (Part Two)
14:31
Load (Part Three)
26:54

As a note, you will want to complete the Extract Code Challenge and Transform Code Challenge 1 before this assignment.

Load Code Challenge 1
02:12

As a note, you will want to complete the Extract Code Challenge and Transform Code Challenge 2 before this assignment.

Load Code Challenge 2
02:09
Our Course Project (Load Homework)
28:05
+ Automating ETL Processes - Transform
11 lectures 02:43:10
Transform (Part One)
21:53
Transform (Part Two)
16:40
Transform (Part Three)
30:38
Transform: (Part Four)
10:05
Transform (Part Five)
20:03
Transform (Part Six)
08:52
Our Course Project (Transform Homework)
16:58

You are required complete all code challenges before the optional practice interview.

Transform Code Challenge 1
01:49

You are required complete all code challenges before the optional practice interview.

Transform Code Challenge 2
02:17
Our Course Project (Transform Homework Part II)
11:58
Transform Coding Live Practice (Optional)
21:57
+ ETL and Performance
3 lectures 42:40
Performance Overview
19:03
Performance Part 1
15:40
Performance Part 2
07:57
+ ETL In the Cloud
1 lecture 16:01

See the resources section for materials listed.

ETL In the Cloud Part 1
16:01
+ Live Practice and Examples
5 lectures 01:12:02

I got a call about an opportunity that someone described as cheap and that I should take advantage of it immediately.  I didn't know if it was or not, so using the tools in this course, I spent a few minutes importing data and doing a quick analysis.  Was it a good opportunity?

Good Deal Or Not? Practice
27:30

While I hope my concerns never come true, ask yourself based on the data I present - is this as good as the real thing?  People may be engaging in a mistake without realizing it.  With these ETL tools, you will speed up your ability to do research like this.

Is This As Good As The Real Thing?
09:29

This video shows excellent analysis by my friend, but with the wrong assumption, a high cost.  I hope all of you can avoid this.  I've experienced this as well and it was a hard lesson learned!  As a bonus, I've attached the CSV in downloadable materials in this lecture.

Be Extremely Careful - Data Example
09:36

What are steps that we can take to get data or assess data when we can't automatically extract data?  Looking at an example of assessing a subjective value (that can't be measured or wasn't measure at the time), we go through seven steps.

How To Handle A Major Extract Challenge?
17:20
What Do the Data Indicate About This Company?
08:07
+ Optional Interview Practice and Reference Requests
1 lecture 05:32
Preparing For the 30 Minute Practice Interview
05:32
+ The Master Class Series
13 lectures 02:52:00
Master Class Series Introduction
02:52
The Master Logic of Extract
08:47
Data Gathering
09:13
Handling Dirty Excel Delimiters
12:58
Another Excel Extract Method
16:35
Handling Wrapped Exceptions
20:00
Handling Useless Characters
06:16
Excel Non-Data Sheets
17:28
Automate Reporting
09:46
Automating Email Loading
18:12
ETL Architecture
20:03
Your ETL Briefcase
10:07
Thinking About ETL In Big Data Environments
19:43
+ Optional Exercises
1 lecture 00:09

The following series is an optional exercise and is not a part of the 5 ETL projects.  Because some environments may require we use other languages, we look at what we can do if we need to swap languages.  In this file, we look at situations where we may be required to swap code or functions into another language.  We also look at a character challenge that's a popular question people ask.

Python Swapping
00:09
Requirements
  • Mandatory: Computer with Microsoft SQL Server Express (or above) 2008R2 (or above)
  • Mandatory: PowerShell ISE 3.0 (or above)
  • Optional: Visual Studio 2010 (or above)
  • Optional: MongoDB
  • Optional: SQL Server Data Tools (SSIS)
Description

This course covers a guide to automating ETL through automation techniques, intended for business owners, freelancers or employees seeking to jump ahead of the competition and seeking to minimize data disasters. With the demand for ETL developers increasing (per the Bureau of Labor Statistics and the opportunity to become a data provider (for business owners), I highlight methods and architecture to minimize the work that must be done for ETL, so that you can spend more time working on enhancing skills. 

I offer a ton of free content on my YouTube channel (SQL In Six Minutes), and suggest students to become familiar with the basics of SQL Server and MongoDB, covered at the below, free playlists: 

  • SQL In Six Minutes (YouTube)

  • The MongoDB 103 (YouTube)

In addition, the only coupon for this course that I offer is available in the trailer for my YouTube channel.  Since this is the most popular question I get asked, see the trailer for the YouTube channel to get it.

This course covers: 

  • Understanding the extract process and constructing architecture to automate it.

  • Understanding the load process, when change is necessary with loading, and how to automate it.

  • Understanding different issues we may face in transform, how to minimize re-facing them with re-usable solutions.

  • Understanding advanced techniques in ETL, the theory behind these, and applications with them.

  • Understanding the basic logic, functions, operators and advanced use of PowerShell so beginners and mid-level developers can advance quickly with it.

  • Completing a large assignment that will have you import over 200K of records.

  • Reviewing case studies of applied ETL techniques, especially useful for freelancers adding credibility to their business.

  • Course features extra ETL content with MongoDB and SSIS/SQL Server Data Tools - though neither of these are required for this course.

To provide maximum learning for students wanting to master ETL skills, whether for business opportunity or employment, the course provides the following material: 

  • New content posted every year since this course began which offer students a look into the new challenges faced by ETL developers.

  • Q&A responses within 48 hours for students who need assistance along with ongoing discussions that students may find helpful.

  • Tri-annual bonus material for long-term and successful students in this course based on several metrics provided by Udemy.  Technology always changes which is why this course continues to be updated and built with rewards for long time students.

  • Focused video lectures with demonstrations of ETL processes in action, the code/script used to make the process occur, and no distracting facial or background movements.

  • Dynamic syllabus with extra content, such as links to articles and videos to enhance and increase your learning, as well as links to specific recruiters in desperate need for ETL developers (students wanting to become employees) and ways to uncover possible data providing opportunities for business owners.

  • Scripts/code that are used in the video lectures, as well as additional tools that students can learn as they construct their own ETL architecture.

  • Course material that we use in the course, such as files we import during the series.

  • Course projects that you complete as well as your own project to develop.

  • Feedback session for students who meet the requirements where you practice answering questions and presenting your own project.

Understanding, manipulating and transforming data will only become more important in the future.  Those who can master this through automation will have an advantage over all their competition because when we see the results of our measurement, only then can we make the appropriate changes; this course shows you how to automate those steps. 

Companies seeking ETL talent.  If you are a company seeking junior to senior ETL developers, you can get in touch with me, as I have metrics on students who completed exceptional work in this course and have expressed interest in working with a firm.  In addition, with some of these students, I've had feedback sessions where I've assessed their skills, helping you identify who may be solid candidates.  You can find my contact information on my YouTube about page, or on my LinkedIn profile.

Who this course is for:
  • Businesses requiring ETL developers
  • Students interested in ETL positions
  • Employees working in ETL positions
  • Employees interested in ETL positions
  • Individual business owners who want to perform ETL on their own
  • Employers needing ETL developers
  • Recruiters wanting to understand ETL positions