Data Engineering from Design to Implementation

Name: Data Engineering from Design to Implementation
Rating: 4.2 (14 reviews)

Learn data engineering from start to finish utilizing best practices and using UML and SQL

Created byHenk van Zyl

Last updated 10/2021

English

What you'll learn

Learn how to design a data model.
Learn how to set up and ETL process from source to database.
Learn how to extract valuable information from your database.
Learn how to present your information as business intelligence in insightful ways.
Learn how to structure data.
Learn how to create database views.
Learn how to conduct calculations in SQL on your data.

Course content

5 sections • 25 lectures • 2h 2m total length

Welcome to the Course!1:12
Introduction to the course and what we will be covering!
Thank you!0:33
Installing PostgreSQL1:52
Download and install PostgreSQL on your Windows machine, choose installation options, set a password, review the default port, and complete the setup.
Installing an IDE (Dbeaver)1:59
Connecting to PostgreSQL1:18

What is an ERD?4:05
Explore how an erd maps a database structure with tables for houses, rooms, and occupants, detailing columns, data types, and one-to-many relationships via crow's feet notation.
What is Normalization?4:15
What is Dimensional Moddeling?5:41
How to create a Star Schema5:35
Explore the star schema, a flat data structure optimized for information delivery in data warehouses, centering analysis on a fact table with dimensions and system generated primary keys.
Data Design Exercise0:57

Database Basics2:51
Loading Data to PostgreSQL13:16
Create a database and a product list table in PostgreSQL, with product_code as the integer primary key, and load csv data into it using copy, then query with select.
Structure of Select Statements3:01
Creating Basic Select Statements12:44
Learn how to write basic select statements and filter data with where, in, not in, and like. Rename columns, order results, limit rows, and format prices for display.

Resource Requirements0:19
Download the two course resources: the cell statuses and the cells grip data sees, to begin the exercises and apply them.
Taking a look at our Data2:00
Creating information packages from our data2:21
Create a product sales information package by identifying dimensions such as customers, products, stores, regions, and sales reps, with facts like quantity and price, and explore flat versus snowflake designs.
Creating a Star Schema from our Information Packages3:46
Creating our Database tables in PostgreSQL11:09
Loading our Temp Tables1:28
Populating our Dimension Tables11:37
Populating our FACT Table8:41
Creating a View to Pull All Information7:58

Requirements

No prior experience needed, this course will cover everything from start(Design) to finish(Implementation)

Description

This course will teach you data engineering from start to finish. This course covers all you need to know from the physical design of our data structures to implementation where we the course will provide you with step by step instructions to follow along.

We will be using unformatted source data and convert this data into usable and valuable information. This process entails everything you need to know, from data design, ETL, information access, data analysis, and lastly information presentation in the form of business intelligence.

This course will include data files that you can use to follow along in the step by step tutorials, to ensure that you have all the information that you need at your disposal to make this course a breeze.

We will be looking at the processes that I have found to work the best in a real live environment. These are methods that can be employed in any business and the teachings of this course will provide you with the required information to start your journey in data engineering.

We will be looking at the following main topics in this course:

Data Design
Data Structuring
Dimensional Modeling
UML (Unified Modeling Language)
ERDs (Entity Relational Models)
SQL (Structured Query Language)
Power BI
etc.

I look forward to seeing you in the course and sharing this learning experience with you!

Who this course is for:

System Engineers
Data Scientists
Data Analysis
Business Analysts
Process Engineers
Solution Architects

Data Engineering from Design to Implementation

What you'll learn

Explore related topics

Course content

Introduction5 lectures • 7min

Data Design5 lectures • 21min

Introduction to SQL (PostgreSQL)4 lectures • 32min

Implementing Data Design and ETL (Extraction, Transformation, and Loading)9 lectures • 49min

Displaying our Information2 lectures • 14min

Requirements

Description

Who this course is for: