Learn Data Engineering

Name: Learn Data Engineering
Rating: 3.7 (57 reviews)

learn to build data Pipeline

Created byAshish Sharma

Last updated 2/2025

English

What you'll learn

Data Engineering
Data Ingestion
Data Storage
Data Transformation
Data Reporting
Data Migration
Debugging Data Pipeline
Reporting
Job Scheduling

Course content

1 section • 8 lectures • 1h 21m total length

Introduction1:06
Kick off your data engineering journey by exploring data ingestion, storage, transformation, migration, and archival, and learn to design efficient data pipelines with a data-first mindset.
Data Ingestion12:22
Learn how data ingestion acquires data from devices, apps, and sensors, driving the first step of data pipelines and establishing the source of truth in cloud or on-premises systems.
Data Storage9:58
Map the data journey from the source database to the application team storage through ingestion. Understand distributed storage, file formats like csv and parquet, and databases and data warehouses.
Data Transformation12:13
Transform raw data into business needs by applying distributed data processing with tools like Hadoop, Spark, and Dataflow, extracting fields and computing experience, and loading transformed data to storage.
Data Reporting9:33
Transform data to business requirements and expose a storage-layer view for reporting teams and customers. Build end-to-end pipelines with layered transformations, storing results in tables, and deliver reports via Tableau.
Scheduling and Archival6:34
Orchestrate layer one to three transformations with a scheduler, load data into storage, and materialize views while archiving processed files under retention policies.
Data Migration18:47
Plan and execute data migration from on-premises to cloud, define scope, map legacy pipelines to cloud services, validate data, and optimize performance for a cost-effective cutover.
Data Debugging10:38
Master data debugging and validation across layered pipelines by backtracking from reports to views, verifying view definitions and column mappings. Automate data validation to ensure consistent data ingestion and reporting.

Requirements

basic Sql

Description

This Course is for the audience who is excited about learning data engineering. It will help you to build data engineering fundamental and mindset. This course Focusses on end to end feature required to build data pipeline. I have covered all the required feature for data engineering. one must be able to think in data pipeline perspective once after completion of this course.

1) Data Engineering : what is data Engineering

2) Ingestion : what is Data Ingestion and how it ingest data.

3) Storage : how to store data and its working

4) Transformation : How data transformation works and different tools available for data Transformation/Manipulation.

5) Data Migration : how to build data pipeline while working on data Migration project . how to map and think different different component at an individual level and build a similar pipeline on a different platform

6) Debugging : how to check and validate any issue in a data pipeline . it will help to learn to identify the root cause of data in any data pipeline.

7) Scheduling and data Archival : how to run different component of a data pipeline using a scheduler.

8) Data Reporting : how data reporting works and develop in a real scenario the publish the client report.

Who this course is for:

This Course is for those who want to develop the mindset of a data Engineer