Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Subscribe
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Learn Data Engineering with Databricks on AWS Cloud
Rating: 3.9 out of 5(52 ratings)
227 students

Learn Data Engineering with Databricks on AWS Cloud

Build Data Engineering Pipelines on AWS using Databricks core features such as Spark and Delta Lake
Created bySKILL CURB
Last updated 9/2023
English

What you'll learn

  • Data Engineering leveraging Databricks features
  • Databricks CLI to manage files, Data Engineering jobs and clusters for Data Engineering Pipelines
  • Deploying Data Engineering applications developed using PySpark on job clusters.
  • Deploying Data Engineering applications developed using PySpark using Notebooks on job clusters
  • Building Data Engineering Pipelines using Spark Structured Streaming on Databricks Clusters
  • Setting up development environment to develop Data Engineering applications using Databricks.

Course content

7 sections • 51 lectures • 4h 50m total length

  • Introduction to Getting Started with Databricks on AWS1:22
  • Signing up for aws free account3:36
  • Logging in into AWS Management Console2:42
  • Setting up Databricks workspace on AWS using Quickstart7:53

    Set up a Databricks workspace on AWS using Quickstart, choosing the premium plan, and start the 14-day trial after creating the stack and workspace in the AWS region.

  • Logging in into Databricks Workspace on AWS3:07
  • Cleaning up the workspace and resources3:20
  • Quick Walkthrough of Databricks UI on AWS2:05
  • Creating Single-Node Databricks cluster on AWS5:34
  • Upload Data using AWS Databricks UI9:00

    Upload data to the Databricks platform on AWS via the UI, create folder hierarchies in DBFS, and infer schema to assign correct data types for tables.

  • Develop spark Application using AWS Databricks Notebook16:00
  • Writing dataframe to DBFS6:44
  • Export and Import AWS Databricks Notebooks4:36

    Export a Databricks notebook from your workspace and import it into another AWS Databricks account using formats such as archive, source file, HTML, or IPython.

Requirements

  • Must have Prior programming experience in SQL and Python.
  • Experience in Spark Programming is a must.
  • Prior basic experience or understanding of cloud services like AWS is important.

Description

Get ready to Learn Data Engineering with Databricks on AWS Cloud with this complete course. Gain familiarity with the course details and topics designed to help you succeed.


This comprehensive course is designed to equip you with the skills and knowledge needed to excel in the field of data engineering using two powerful platforms: Databricks and Amazon Web Services (AWS). Data engineering is the backbone of any successful data-driven initiative, and Databricks, a unified analytics platform, has emerged as a leading choice for data engineers and data scientists worldwide. When combined with AWS, a cloud computing powerhouse, you have a robust ecosystem that can handle data at scale, provide advanced analytics capabilities, and support a wide range of data sources and formats.


Learn about Data Engineering with Databricks on AWS with Hands-On Labs

Learn Data Engineering with Databricks on AWS Cloud is a hands-on practice course designed to familiarize you with the core functionality of Databricks by connecting it with AWS to perform Data Engineering. Through hands-on exercises, you'll gain a thorough understanding of Databrick's architecture and how it revolutionizes data engineering in the cloud. You'll explore the seamless integration of Databricks with AWS services, such as Amazon S3 and Glue, unlocking a world of possibilities for managing and analyzing your data.

This course has been meticulously designed to provide you with both a solid theoretical foundation and extensive hands-on practice in the dynamic realms of data engineering, Databricks, and Amazon Web Services (AWS).

The course comprises approximately 50 labs starting from the basics and moving to high levels in terms of complexity.

Who should take this course?

The course "Learn Data Engineering with Databricks on AWS Cloud" is designed for a wide range of individuals who are interested in building expertise in data engineering using Databricks on the AWS Cloud. If you're looking to start a career in data engineering, this course is an excellent choice. It will provide you with the foundational knowledge and practical skills needed to become a successful data engineer. Data scientists and analysts who want to expand their skill set and be able to work with large-scale data processing, data pipelines, and data lakes can greatly benefit from this course. IT professionals who want to transition into roles focused on data engineering and cloud computing can use this course as a stepping stone to acquire the necessary skills and knowledge. Individuals interested in cloud computing, specifically AWS, and its applications in data engineering will gain a deep understanding of cloud-based data engineering solutions.

Requirements

● Basic knowledge of SQL or writing queries in any language

● Scripting in Python Willingness to explore, learn, and put in the extra effort to succeed

● An active AWS Account & know-how of basic cloud fundamentals

● Programming experience using Python

● Data Engineering experience using Spark


Who this course is for:

  • Software engineers, aspiring data engineers or data analyst & data scientists
  • Programmers and Database Administrators with experience in writing SQL queries
  • BI Analysts looking to enhance their understanding of data engineering, particularly in the context of big data and cloud platforms, can leverage this course to broaden their skill set.
  • IT professionals who want to transition into roles focused on data engineering and cloud computing

Report abuse