Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS CompTIA Security+ AWS Certified Developer - Associate
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Personal Development Mindfulness Personal Transformation Meditation Life Purpose Coaching Neuroscience
Web Development JavaScript React CSS Angular PHP WordPress Node.Js Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Retargeting
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Analysis Data Modeling Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Freelancing Blogging Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
2021-01-07 00:44:15
30-Day Money-Back Guarantee

This course includes:

  • 6.5 hours on-demand video
  • 3 articles
  • 3 downloadable resources
  • 2 practice tests
  • Full lifetime access
  • Access on mobile and TV
Development Database Design & Development Microsoft Azure

Microsoft Azure Data Lake Storage Service (Gen1 & Gen2)

How to ingest, process and export data in Azure Data Lake using Databricks and HDInsight,
Rating: 4.6 out of 54.6 (183 ratings)
5,315 students
Created by Eshant Garg
Last updated 1/2021
English
English [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • Microsoft Azure Data Lake
  • Different tools and scenarios to ingest data in to Data Lake
  • Data Lifecycle and Architecture around Data Lake
  • Use Azure Databricks and HDInsight to process data in ADLS
  • Five layers of security to protect Data Lake
  • Monitor the performance of your Data lake
  • Data Lake optimization Strategy
Curated for the Udemy for Business collection

Course content

10 sections • 63 lectures • 6h 17m total length

  • Preview02:35
  • Before you start...
    01:29
  • Demo Resources and PPT
    00:01

  • Create Azure free subscription
    06:13
  • Azure Portal Overview
    14:45
  • Azure Services Overview
    11:08
  • Resource Management group and Subscription
    08:34
  • Resource Groups
    08:18
  • Tagging
    03:09
  • Delete resources and set budget
    06:59

  • Preview04:07
  • Preview03:56
  • Preview01:34
  • Preview05:17
  • Azure Data Lake vs Azure Blob storage
    04:37
  • Provision Azure Data Lake Gen2 Account
    11:01
  • Preview10:16
  • Hierarchical namespace
    06:12
  • Other Data Lake Gen 2 features
    06:00

  • Tools to Ingest Data in Data Lake
    11:03
  • Demo - Ingest using Portal and SE
    09:11
  • Preview06:04
  • Demo: Azure Blob storage to Data Lake Gen2 using Data Factory
    09:04
  • Demo: SQL Server to Data Lake Gen2 using Data Factory
    14:13
  • Demo: Amazon S3 to Data Lake Gen2 using Data Factory
    08:16

  • Data flow around Data Lake
    04:52
  • Data Lake and Transient clusters
    02:13

  • Imp NOTE before you start Demo
    00:50
  • Demo overview
    04:35
  • Demo: Provision Databricks, Clusters and workbook
    11:22
  • Demo: Mount Data Lake to Databricks DBFS
    12:24
  • Demo: Explore, Analyze, Clean, Transform and Load Data
    18:04

  • Demo overview
    01:00
  • Create Azure Data Lake Storage Gen 2 (source) and SQL Server (destination)
    05:31
  • What is Managed Identity
    05:26
  • Add Managed Identity to Gen2 and Database accounts
    02:46
  • Create HDInsight Interactive Query Cluster
    07:09
  • Ambari overview and UI
    06:13
  • Ingest dataset in to DataLake storage
    04:52
  • Data Extraction with Hive
    05:02
  • Data transformation with Hive
    02:33
  • Data Export using Sqoop
    03:44
  • Summary
    00:38

  • Introduction
    02:01
  • Storage Access Keys
    03:24
  • SAS - Shared Access Signature
    07:16
  • Azure Active Directory
    04:44
  • Access Control List (ACL)
    06:23
  • Firewalls and Virtual Networks
    03:53
  • Encryption in Transit
    03:31
  • Encryption at Rest
    02:36
  • Advanced threat protection
    02:06

  • Activity Log
    04:41
  • Demo Activity Logs
    06:42
  • Metrics
    03:38
  • Demo: Metrics
    08:47
  • Demo: Insights
    08:26
  • Demo: Alerts
    08:25
  • Diagnostic settings
    08:16
  • Demo: Diagnostic settings
    08:20
  • Optimization
    05:39

  • Delete Resources
    04:08
  • Practice Test 1
    5 questions
  • Practice Test 2
    7 questions
  • Bonus Lecture
    00:47

Requirements

  • Basic Database concepts

Description

Course Introduction

  • Azure Data Lake Storage Gen2 (ADLS) is a cloud-based repository for both structured and unstructured data. For example, you could use it to store everything from documents to images to social media streams.

  • The most effective way to do big data processing on Azure is to store your data in ADLS and then process it using Spark (which is essentially a faster version of Hadoop) on Azure Databricks.

  • In this course, you will follow hands-on examples to import data into ADLS and then securely access it and analyze it using Azure Databricks and Azure HDInsight.

  • You will also learn how to monitor and optimize your Data Lake Storage.

Expected Outcomes

After this course:

  • You will have a very good understanding of Data lake features and properties

  • You will learn about different tools and scenarios to ingest data into Data Lake.

  • You will learn about Data Lifecycle and Architecture around Data Lake

  • You will be doing end to end demos to ingest, process, and export data using Databricks and HDInsight.

  • You will learn about 5 layers of Data Security and how to configure them using the Azure portal.

  • You will also learn about different tools Azure provides to monitor Data Lake Storage service.

  • And finally, you will learn optimization techniques for Data Lake Storage.

Intended Audience

  • Anyone interested in learning Azure Data Lake Storage Service

Level

  • Beginners and intermediate level

  • I have a few crash courses (Free) for absolute beginners, you can find links on my website.

Prerequisites

  • There is no prerequisite for this course

  • Basic understanding of Data warehouse and Database, in general, will help you understand quickly.

Language

  • English

  • If you are not comfortable in English, please do not take a course, captions are not good enough to understand the course.

What's inside

  • Video lectures, PPTs, Demo Resources, Quiz, Assignment, other important links

  • Full lifetime access with all future updates

  • Certificate of course completion

  • 30-Day Money-Back Guarantee

Some students Feedback

  • One of the most amazing courses i have ever taken on Udemy. Please don't hesitate to take this course. The instructor is really professional and has a great experience about the subject of the course. - Khadija Badary

  • Very nicely explained most of the concepts. a must have course for beginners - Manoranjan Swain

  • I appreciate this course explaining everything in great detail for a beginner. This will assist me in overcoming challenges at my work - Benjamin Curtis

  • Good course for Beginners. Labs are really helpful to grasp the concept. Thank you - Sapna

Who this course is for:

  • Anyone interested in Azure’s big data analytics services
  • Microsoft Azure Data Engineers
  • Database and BI developers
  • Database Administrators
  • Data Analyst or similar profiles

Instructor

Eshant Garg
Azure Data Architect || Author || Adviser
Eshant Garg
  • 4.6 Instructor Rating
  • 6,355 Reviews
  • 39,970 Students
  • 12 Courses

13 years of extensive professional experience with expertise in Microsoft Database and Business Intelligence Solutions, Advanced Analytics, Reporting and Azure cloud computing technologies

CERTIFICATIONS

- DP-200: Implementing an Azure Data Solution
- DP-201: Designing an Azure Data Solution
- Microsoft Certified Technology specialist (MCTS) – SQL Server Database development
- Information Technology Infrastructure Library (ITIL V3) Certified

SPECIALITY

Azure : Data Lake, Data Factory, Synapse Analytics (DW), PolyBase, Stream Analytics & Storage

Big Data Tech : HDInsgiht, Databricks, CosmosDB, Hadoop, Spark, PySpark, Hive, Sqoop

Database versions : Microsoft SQL Server 2005-2016, NoSQL

Languages : T-SQL, USQL, Python, MDX, DAX, PySpark

BI Tools : SSRS, SSIS, SSAS, Power BI


I have been messing around with data for more than a decade now, I love to work on Database internals, performance tuning, and optimization, Database design, modeling, and implementation especially using Microsoft SQL Server and Azure database and analytics technologies.

As a developer and architect, I have worked closely with customers, users, and colleagues to support business solutions across a variety of industries including healthcare, insurance, finance, and government ranging from small companies to fortune 500 companies. These experiences provided opportunities to see varied implementations, troubleshoot performance-related problems, give guidance on design, and provide consultation on optimization, availability, and recoverability.

I helped in End to End Implementations of many database projects, providing team leadership and project management, requirements gathering, specification documentation, functional/technical design, Data modeling & solution architecture.

Outside of the technical world, I love yoga and meditation. I am a student of ancient yogic text Bhagavad Gita and love to discuss and practice philosophical teachings.

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.