PySpark - Build DataFrames with Python, Apache Spark and SQL

Name: PySpark - Build DataFrames with Python, Apache Spark and SQL
Rating: 4.4 (32 reviews)

Build an amazing DataFrames with Python, Apache Spark, and SQL

Created byMammoth Interactive, John Bura

Last updated 2/2021

English

What you'll learn

Create big data streaming pipelines with Spark using Python
Learn how to apply Spark on distributed Dataframes
Use Python with Big Data on a distributed framework (Apache Spark)

Course content

4 sections • 35 lectures • 5h 56m total length

Project Preview2:33
What Is Apache Spark2:37
What Are Resilient Distributed Datasets1:08
Explore resilient distributed datasets, the fundamental immutable, distributed data structure in Apache Spark. Learn that RDDs can hold any type, are partitioned and computed on multiple nodes to boost performance.
What Is A Dataframe1:47
What You'll Need1:47
Learn what a data frame is and how Spark optimizes PySpark DataFrames for large, structured data; explore formats, sources, and language support before coding with Weisbach.
Source Files

Intro To Course And Python9:57
Explore Python basics, from variables, operators, conditionals, and loops to functions and classes and objects, all taught from a beginner-friendly browser-based setup using Google Colab.
Variables19:19
Type Conversion Examples10:06
Operators28:54
Collections8:24
List Examples19:41
Tuples Examples8:36
Dictionaries Examples14:26
Ranges Examples8:32
Conditionals6:43
Explore conditionals in Python by learning how if statements, elif, and else control the program flow, test variable states, and handle complex conditions with and/or and nesting.
If Statement Examples21:32
Loops29:42
Functions17:01
Parameters And Return Values Examples13:54
Explore Python function basics by using parameters and return values, adding default parameters, and implementing bounds checks to move a position from start 0 to end 10.
Classes And Objects34:11
Explore classes and objects in Python by building a game character, learning how classes define blueprints, instantiate objects, use fields and methods, and apply inheritance and static members.
Inheritance Examples17:29
Static Members Examples11:05
Summary And Outro4:08
Revisit the Python language basics by reviewing variables, collections, conditionals, loops, functions, and classes, and encourage practice and exploration of libraries like pandas for data analysis and machine learning.

Requirements

No necessary experience needed

Description

This course covers all the fundamentals about Apache Spark streaming with Python and teaches you everything you need to know about developing Spark streaming applications using PySpark, the Python API for Spark. At the end of this course, you will gain in-depth knowledge about Spark streaming and general big data manipulation skills to help your company to adapt Spark Streaming for building big data processing pipelines and data analytics applications. This course will be absolutely critical to anyone trying to make it in data science today.

Spark can perform up to 100x faster than Hadoop MapReduce, which has caused an explosion in demand for this skill! Because the Spark 2.0 DataFrame framework is so new, you now have the ability to quickly become one of the most knowledgeable people in the job market!

This course will teach the basics with a crash course in Python, continuing on to learning how to use Spark DataFrames with the latest Spark 2.0 syntax! Once we've done that we'll go through how to use the MLlib Machine Library with the DataFrame syntax and Spark. All along the way, you'll have exercises and Mock Consulting Projects that put you right into a real-world situation where you need to use your new skills to solve a real problem!

We also cover the latest Spark Technologies, like Spark SQL, Spark Streaming, and advanced models like Gradient Boosted Trees! After you complete this course you will feel comfortable putting Spark and PySpark on your resume! This course also has a full 30-day money-back guarantee and comes with a LinkedIn Certificate of Completion!

If you're ready to jump into the world of Python, Spark, and Big Data, this is the course for you!

Who this course is for:

Someone who knows Python and would like to learn how to use it for Big Data
Someone who is very familiar with another programming language and needs to learn Spark
Developers transferring from other languages
Python Developers looking to get better at Data Streaming
Spark Developers eager to expand their skills.

PySpark - Build DataFrames with Python, Apache Spark and SQL

What you'll learn

Explore related topics

Course content

00 Course Overview6 lectures • 10min

01 Mammoth Interactive Courses Introduction3 lectures • 15min

02 Introduction to Python (Prerequisite)18 lectures • 4hr 44min

03 Build DataFrames from Spreadsheets with PySpark8 lectures • 48min

Requirements

Description

Who this course is for: