
Explore how data lake architecture relates to Hadoop, recognizing Hadoop as one component, not the sole data platform. See complementary options like Apache Kafka for real-time data.
Explore the Azure Data Lake Gen2 storage account overview, covering resource group, location, tags, storage types, metrics, activity log, access control, data transfer, and lifecycle management.
Enable hierarchical namespace in azure data lake storage gen2 creates a true directory tree, enabling fast folder operations and seamless Hadoop integration for improved analytics.
See how data lake design with separate storage and compute enables cost savings and on-demand analytics via transient clusters, keeping data safe while you power down unused resources.
Provision a Databricks workspace and a premium database service with role-based access control, then create and attach an interactive cluster to a notebook and to a workbook.
Mount the data lake to Databricks DBFS by creating a service principal, granting read, write, and execute access, and configuring the mount with the app client and directory IDs.
Learn how access control lists manage read, write, and execute permissions for users, groups, and service principals in Azure Data Lake Storage, with inheritance and group-based best practices.
Explore how to lock down Azure storage using firewalls and virtual networks, restricting access to selected IP ranges and virtual networks, with exceptions for trusted Microsoft services and logging access.
Demonstrate encryption at rest as a built-in, mandatory feature for all storage accounts, using 256-bit Microsoft managed keys by default, with optional customer-managed keys via key vault.
Explore creating alerts in an Azure storage account, define metric and signal-based conditions, and use action groups to email or automate responses when capacity or security events exceed thresholds.
Course Introduction
Azure Data Lake Storage Gen2 (ADLS) is a cloud-based repository for both structured and unstructured data. For example, you could use it to store everything from documents to images to social media streams.
The most effective way to do big data processing on Azure is to store your data in ADLS and then process it using Spark (which is essentially a faster version of Hadoop) on Azure Databricks.
In this course, you will follow hands-on examples to import data into ADLS and then securely access it and analyze it using Azure Databricks and Azure HDInsight.
You will also learn how to monitor and optimize your Data Lake Storage.
Expected Outcomes
After this course:
You will have a very good understanding of Data lake features and properties
You will learn about different tools and scenarios to ingest data into Data Lake.
You will learn about Data Lifecycle and Architecture around Data Lake
You will be doing end to end demos to ingest, process, and export data using Databricks and HDInsight.
You will learn about 5 layers of Data Security and how to configure them using the Azure portal.
You will also learn about different tools Azure provides to monitor Data Lake Storage service.
And finally, you will learn optimization techniques for Data Lake Storage.
Intended Audience
Anyone interested in learning Azure Data Lake Storage Service
Level
Beginners and intermediate level
I have a few crash courses (Free) for absolute beginners, you can find links on my website.
Prerequisites
There is no prerequisite for this course
Basic understanding of Data warehouse and Database, in general, will help you understand quickly.
Language
English
If you are not comfortable in English, please do not take a course, captions are not good enough to understand the course.
What's inside
Video lectures, PPTs, Demo Resources, Quiz, Assignment, other important links
Full lifetime access with all future updates
Certificate of course completion
30-Day Money-Back Guarantee
Some students Feedback
One of the most amazing courses i have ever taken on Udemy. Please don't hesitate to take this course. The instructor is really professional and has a great experience about the subject of the course. - Khadija Badary
Very nicely explained most of the concepts. a must have course for beginners - Manoranjan Swain
I appreciate this course explaining everything in great detail for a beginner. This will assist me in overcoming challenges at my work - Benjamin Curtis
Good course for Beginners. Labs are really helpful to grasp the concept. Thank you - Sapna