Hadoop & Spark

Name: Hadoop & Spark
Rating: 4.2 (93 reviews)

Master the Hadoop ecosystem and Spark tutorial

Created byTani Das

Last updated 2/2021

English

What you'll learn

Hadoop & Spark

Course content

1 section • 29 lectures • 26h 50m total length

Hadoop 1 - Big data intro11:57
Hadoop 2 - Hadoop Intro48:55
Hadoop 3 - Features39:31
Hadoop Architecture 1 - Servers1:17:46
Hadoop Architecture 2 - Daemon And Name node47:19
Hadoop Architecture 3 - Job Tracker1:21:11
Hadoop Architecture 4 - Load Balance51:32
Hadoop Architecture 5 - Namenode1:07:17
Hadoop Architecture 6 - Yarn51:09
MapReduce 1 - Mapper48:12
MapReduce 2 - Reducer49:17
MapReduce 343:09
MapReduce 443:52
MapReduce 5 - Mapper Only Functionality1:00:21
Demonstrate mapper-only mapreduce workflows by applying filters, transformations, and column selections on unstructured data, while suspending reducers and producers when no aggregation is needed.
MapReduce 6 - Reducer Req31:37
Spark 1 - Introduction & Advantages50:17
Spark 2 - Spark Components1:07:48
Spark 3 - Architecture1:01:49
Spark 4 - Rdd, Caching And Persistence56:51
Spark Prog 1 - Create Rdd And Aggregations1:09:45
Learn how to create RDDs in Spark from local data and text files, apply map, flatMap, filter, and reduce, and perform a word count with partitioned aggregations.
Spark Prog 2 - Aggregation1:00:11
Spark Prog 3 - Multiple Aggregation50:50
Spark prog 3 shows how to perform multiple aggregations with grouping and multi-level grouping, applying max, min, average, sum, and count across data.
Spark Prog 4 - Merging And Cross Joints1:08:47
Demonstrates merging datasets with unions, handling duplicates with distinct, and applying Cartesian products via cross joins to compare data across departments and sales in Spark.
Scala 1 - Introduction1:08:07
Scala 2 - Map Collection20:48
Learn Scala map collections by transforming elements, accessing keys and values, and filtering with boolean conditions using map and flatMap, including uppercase transformations and concise shortcuts.
Scala 3 - Transformation Over Collections53:46
Scala 4 - Perform Transformations1:20:16
Scala 5 - Function, Case Classes1:26:45
Scala 61:01:23

Requirements

Description

The world of Hadoop and "Big Data" can be intimidating - hundreds of different technologies with cryptic names form the Hadoop ecosystem. With this Hadoop tutorial, you'll not only understand what those systems are and how they fit together - but you'll go hands-on and learn how to use them to solve real business problems! “Big data" analysis is a hot and highly valuable skill – and this course will teach you two technologies fundamental to big data quickly: MapReduce and Hadoop

Understanding Hadoop is a highly valuable skill for anyone working at companies with large amounts of data.

Almost every large company you might want to work at uses Hadoop in some way, including Amazon, Ebay, Facebook, Google, LinkedIn, IBM, Spotify, Twitter, and Yahoo! And it's not just technology companies that need Hadoop; even the New York Times uses Hadoop for processing images.

Also, you will learn about how Spark works best when using the Scala programming language, and this course also includes Scala to get you up to speed quickly.

The course is aimed at Software Engineers, Database Administrators, and System Administrators that want to learn about Big Data. Other IT professionals can also take this course, but might have to do some extra research to understand some of the concepts. Once you complete the it, you'll walk away from this course with a real, deep understanding of Hadoop and its associated distributed systems, and you can apply Hadoop to real-world problems.

Below are the topics discussed:-

Part 1 :- Introduction

Part 2 :- Hadoop Architecture

Part 3 :- MapReduce

Part 4 :- Spark

Part 5 :- Spark Programming

Part 6 :- Scala

Who this course is for:

Hadoop & Spark