Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Data Lake Fundamentals: Key Concepts and Best practices

Name: Data Lake Fundamentals: Key Concepts and Best practices
Rating: 4.0 (2 reviews)

Key concepts and fundamentals of Data Lake implementation

Created byLuiz Fabiano de Souza

Last updated 7/2024

English

What you'll learn

Learn the fundamentals of a Data Lake
How to manage a Lambda architecture in a Data Lake
Master the fundamentals of all the layers in a Data Lake implementation
Learn to add value to your company with a Data Lake
Study the core architecture principles of a Data Lake

Course content

3 sections • 16 lectures • 42m total length

Course Introduction3:30
In this video, we will make the course introduction.
What is a Data Lake?1:58
This class will teach you what is a Data Lake.
Why should I learn about it?1:20
Here is the explanation of why this course is so important.

Characteristics of a Data Lake2:32
This class explain the main characteristics of a Data Lake.
How does a Data Lake adds Value2:32
In this class you will know how a data lake can add value to your company.
Data Lake Vs Data Warehouse3:54
In this class we will explain the differences between data lake and data warehouses.
Data Lake Challenges3:01
Do you know what are the challenges today for the big data companies? Here we will discuss some challenges.

Core Architecture Principles3:01
We will review some core architecture principles of a data lake.
Lambda Architecture-Driven Data Lake1:40
Lambda architecture is a way of processing massive quantities of data (i.e. "Big Data") that provides access to batch-processing and stream-processing methods with a hybrid approach.
Data Ingestion Layer3:35
Raw data layer and also called the Ingestion Layer/Landing Area, because it is literally the sink of our Data Lake. The main objective is to ingest data into Raw as quickly and as efficiently as possible. To do so, data should remain in its native format. We don't allow any transformations at this stage.
Batch Speed Layer2:36
This layer handles the data that are not already delivered in the batch view due to the latency of the batch layer. In addition, it only deals with recent data in order to provide a complete view of the data to the user by creating real-time views.
Storage Layer1:53
A data lake is a central location that holds a large amount of data in its native, raw format. Compared to a hierarchical data warehouse, which stores data in files or folders, a data lake uses a flat architecture and object storage to store the data.
Serving Layer2:36
The core task of the serving layer is to expose the views created by both the batch and speed layer for querying by other systems or users. Apart from this, there is a good amount of orchestration work done by this layer.
Data Acquisition Layer2:50
The data acquisition layer allows data to be physically integrated into the Data Lake. The layer receives the data from the source and distributes it in the data lake system.
Messaging Layer2:54
In this class we will learn about the messaging layer.
The messaging layer in Data Lake takes care of some functions/capabilities: One of the core capabilities of this layer is it's ability to decouple both the source (producer) and destination (consumer).
Object Storage or HDFS2:48
In this class you will learn the difference between Object storage system vs HDFS.

Requirements

Minimum Data Science structure knowledge required

Description

Hello! Welcome to the "Data Lake Fundamentals" course!!

Did you know companies generate massive amounts of data every year? Data that, if used correctly, can transform businesses.

Traditional data management solutions struggle with today's data volumes. Data Lakes are the modern solution, offering a way to store, manage, and analyze all types of data efficiently

A Data Lake is a centralized repository that stores raw data in its native format, whether structured, semi-structured, or unstructured. It provides the flexibility needed for advanced data analysis."

Data Lakes offer scalability, cost efficiency, and support for IoT and regulatory compliance. Companies using Data Lakes turn data into valuable insights, driving innovation and gaining a competitive edge.

In our 'Data Lake Fundamentals' course, you'll learn:

-What a Data Lake is.

-Key features and benefits.

-How to architect and govern a Data Lake.

-Practical use cases and real-world examples

This course has three parts:

Part I: Course Introduction

In this section we can understand clearly what is a Data Lake and why they are so important.

Classes:

Course Introduction
What is a Data Lake?
Why should I learn about it?

Part II - Characteristics and Comparisson of current scenario

In this section we will deep dive in the concepts and comparisson between traditional warehouses and Data Lakes. Also, we will understand some challenges for implementation.

Classes:

Characteristics of a Data Lake
How does a Data Lake adds value?
Data Lake Vs Data Warehouse
Data Lake challenges

Part III - Lamda Architecture implementation

Here we will understand how a lambda architecture data lake is splitted.

Classes:

Core architecture principles
Lambda architecture-drive Data Lake
Data Ingestion layer
Batch speed layer
Storage layer
Serving layer
Data Acquisition layer
Messaging layer
Object storage vs HDFS

Who this course is for:

Data engineers
Data analysts
Software engineers
Python developers

Data Lake Fundamentals: Key Concepts and Best practices

What you'll learn

Explore related topics

Course content

Part I - Course Introduction3 lectures • 7min

Part II - Characteristics and Comparisson of current scenario4 lectures • 12min

Part III9 lectures • 24min

Requirements

Description

Who this course is for: