Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS CompTIA Security+ AWS Certified Developer - Associate
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Personal Development Mindfulness Meditation Personal Transformation Life Purpose Emotional Intelligence Neuroscience
Web Development JavaScript React CSS Angular PHP WordPress Node.Js Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Google Analytics
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Modeling Data Analysis Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Blogging Freelancing Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee
Development Database Design & Development Apache Spark

Delta Lake with Apache Spark using Scala

Delta Lake with Apache Spark using Scala on Databricks platform
Rating: 2.6 out of 52.6 (28 ratings)
161 students
Created by Bigdata Engineer
Last updated 10/2020
English
English [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • You will be able to learn Delta Lake with Apache Spark in few hours
  • Basics to Advance Level of Knowledge about Delta Lake
  • Hands on practice with Delta Lake
  • You will Learn Delta Lake with Apache Spark using Scala on DataBricks Platform
  • Learn how to leverage the power of Delta Lake with a Spark Environment!
  • Learn about the DataBricks Platform!

Course content

1 section • 50 lectures • 1h 56m total length

  • Preview03:21
  • Preview01:30
  • Introduction to Data Lake
    01:09
  • Key Features of Delta Lake
    04:57
  • Introduction to Spark
    04:04
  • Free Account creation in Databricks
    01:51
  • Provisioning a Spark Cluster
    02:15
  • Basics about notebooks
    07:29
  • Dataframes
    04:47
  • (Hands On) Create a table
    06:38
  • (Hands On) Write a table
    14:12
  • Preview06:52
  • Schema validation
    02:50
  • (Hands On) Update table schema
    03:01
  • Table Metadata
    01:53
  • Delete from a table
    01:44
  • Update a Table
    02:11
  • Vacuum
    01:59
  • History
    01:34
  • Concurrency Control
    01:08
  • Optimistic concurrency control
    02:33
  • Migrate Workloads to Delta Lake
    05:23
  • Optimize Performance with File Management
    01:13
  • FAQ (Interview Question on Optimization) 1
    01:47
  • FAQ (Interview Question on Optimization) 2
    01:50
  • FAQ (Interview Question on Optimization) 3
    00:51
  • Auto Optimize
    02:45
  • FAQ (Interview Question on Auto Optimize) 4
    00:50
  • FAQ (Interview Question on Auto Optimize) 5
    01:06
  • Optimize Performance with Caching
    01:11
  • Delta and Apache Spark caching
    03:26
  • Cache a subset of the data
    01:37
  • Isolation Levels
    01:06
  • Best Practices
    02:56
  • FAQ (Interview Question) 6
    01:06
  • FAQ (Interview Question) 7
    00:37
  • FAQ (Interview Question) 8
    00:42
  • FAQ (Interview Question) 9
    00:20
  • FAQ (Interview Question) 10
    00:26
  • FAQ (Interview Question) 11
    00:28
  • FAQ (Interview Question) 12
    00:27
  • FAQ (Interview Question) 13
    00:43
  • FAQ (Interview Question) 14
    00:55
  • FAQ (Interview Question) 15
    01:39
  • FAQ (Interview Question) 16
    00:31
  • FAQ (Interview Question) 17
    00:32
  • FAQ (Interview Question) 18
    01:00
  • FAQ (Interview Question) 19
    01:25
  • Important Lecture
    00:20
  • Bonus Lecture
    00:52

Requirements

  • Apache Spark and Scala and SQL basic knowledge is necessary for this course

Description

You will Learn Delta Lake with Apache Spark using Scala on DataBricks Platform


Learn the latest Big Data Technology - Spark! And learn to use it with one of the most popular programming languages, Scala!

One of the most valuable technology skills is the ability to analyze huge data sets, and this course is specifically designed to bring you up to speed on one of the best technologies for this task, Apache Spark! The top technology companies like Google, Facebook, Netflix, Airbnb, Amazon, NASA, and more are all using Spark to solve their big data problems!

Spark can perform up to 100x faster than Hadoop MapReduce, which has caused an explosion in demand for this skill! Because the Spark 3.0 DataFrame framework is so new, you now have the ability to quickly become one of the most knowledgeable people in the job market!


Delta Lake is an open-source storage layer that brings reliability to data lakes. Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs.


Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.


Topics Included in the Courses

  • Introduction to Delta Lake

  • Introduction to Data Lake

  • Key Features of Delta Lake

  • Introduction to Spark

  • Free Account creation in Databricks

  • Provisioning a Spark Cluster

  • Basics about notebooks

  • Dataframes

  • Create a table

  • Write a table

  • Read a table

  • Schema validation

  • Update table schema

  • Table Metadata

  • Delete from a table

  • Update a Table

  • Vacuum

  • History

  • Concurrency Control

  • Optimistic concurrency control

  • Migrate Workloads to Delta Lake

  • Optimize Performance with File Management

  • Auto Optimize

  • Optimize Performance with Caching

  • Delta and Apache Spark caching

  • Cache a subset of the data

  • Isolation Levels

  • Best Practices

  • Frequently Asked Question in Interview


About Databricks:

Databricks lets you start writing Spark code instantly so you can focus on your data problems.

Who this course is for:

  • Beginner Apache Spark Developer, Bigdata Engineers or Developers, Software Developer, Machine Learning Engineer, Data Scientist, Data Analyst, Analyst

Instructor

Bigdata Engineer
Bigdata Engineer
Bigdata Engineer
  • 3.4 Instructor Rating
  • 264 Reviews
  • 15,305 Students
  • 20 Courses

I am Solution Architect with 12+ year’s of experience in Banking, Telecommunication and Financial Services industry across a diverse range of roles in Credit Card, Payments, Data Warehouse and Data Center programmes

My role as Bigdata and Cloud Architect to work as part of Bigdata team to provide Software Solution.

Responsibilities includes,

- Support all Hadoop related issues
- Benchmark existing systems, Analyse existing system challenges/bottlenecks and Propose right solutions to eliminate them based on various Big Data technologies
- Analyse and Define pros and cons of various technologies and platforms
- Define use cases, solutions and recommendations
- Define Big Data strategy
- Perform detailed analysis of business problems and technical environments
- Define pragmatic Big Data solution based on customer requirements analysis
- Define pragmatic Big Data Cluster recommendations
- Educate customers on various Big Data technologies to help them understand pros and cons of Big Data
- Data Governance
- Build Tools to improve developer productivity and implement standard practices

I am sure the knowledge in these courses can give you extra power to win in life.

All the best!!

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Impressum Kontakt
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.