Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Big Data Hadoop and Spark with Scala
Rating: 4.5 out of 5(1,041 ratings)
39,326 students

Big Data Hadoop and Spark with Scala

Complete course (No Prerequisites) - Big Data Hadoop with Spark and Eco system
Created byHarish Masand
Last updated 11/2023
English
English [Auto],

What you'll learn

  • You will learn about Hadoop, eco system, tools and spark
  • Big Data Hadoop Development

Course content

4 sections • 63 lectures • 44h 44m total length

  • Introduction to Data Engineering Career Path1:46
  • A1 - Hadoop Intro (1/2)34:48
  • A2 - Hadoop Intro (2/2)52:50
  • Materials0:03
  • A3 - Python (1/4) Practical26:35
  • A4 - Python (2/4) Practical27:23

    Master Python basics from variable naming and whitespace to indentation in if statements, plus strings, data types, indexing, slicing, and basic formatting.

  • A5 - Python (3/4) Practical1:00:35
  • A6 - Python (4/4) Practical40:09
  • A7 - Linux (1/2) Practical30:02
  • A8 - Linux (2/2) Practical1:15:16
  • OPTIONAL - Installing Cloudera VM on AWS EC2 Cloud1:17:56

    NOTE: This is PURELY OPTIONAL , for those people, for whom ORACLE VM is not working on there local laptop for RAM shortage, or virtualization issues or other reasons.


    i will suggest atleast do watch, it will definitely add some additional knowledge.

  • A9 - HDFS (1/4)51:16

    Explore the origin and architecture of HDFS, including the master and data nodes, block storage, replication, metadata, and how MapReduce and YARN manage data processing and resources.

  • A10 - HDFS (2/4)21:45
  • A11 - HDFS (3/4) Practical42:19

    Explores HDFS basics, name node, and secondary name node, and block storage across data nodes; introduces YARN with resource manager, application master, node manager, and containers for MapReduce and Spark processing.

  • A12 - HDFS (4/4) Practical20:47
  • A13 - Map Reduce (1/4)32:25

    Explore MapReduce basics: parallel data processing on commodity hardware with map and reduce phases, key-value pairs, and shuffle and sort, using a word-count example in Hadoop.

  • A14 - Map Reduce (2/4)47:14
  • A15 - Map Reduce (3/4) Practical44:23
  • A16 - Map Reduce (4/4) Practical55:50

    Explore a practical map reduce workflow to identify the most frequent word in a text. Emit word counts with map, lowercase, group by word, and reduce to final tallies.

  • A17 - Pig (1/4)29:11
  • A18 - Pig (2/4) Practical13:34
  • A19 - Pig (3/4) Practical1:14:26
  • A20 - Pig (4/4) Practical46:05
  • A21 - Hive (1/4)49:21
  • A22 - Hive (2/4) Practical58:28
  • A23 - Hive (3/4) Practical51:04
  • A24 - Hive (4/4) Practical1:10:56

Requirements

  • No Pre-requisite required.
  • Curiosity to learn new technology.

Description

This course will make you ready to switch career on big data hadoop and spark.

After this watching this, you will understand about Hadoop, HDFS, YARN, Map reduce, python, pig, hive, oozie, sqoop, flume, HBase, No SQL, Spark, Spark sql, Spark Streaming.


This is the one stop course. so dont worry and just get started. 

You will get all possible support from my side.

For any queries, feel free to message me here.


Note: All programs and materials are provided.


About Hadoop Ecosystem, NoSQL and Spark:

Hadoop and its Ecosystem: Hadoop is an open-source framework for distributed storage and processing of large data sets. Its core components include the Hadoop Distributed File System (HDFS) for data storage and the MapReduce programming model for data processing. Hadoop's ecosystem comprises various tools and frameworks designed to enhance its capabilities. Notable components include Apache Pig for data scripting, Apache Hive for data warehousing, Apache HBase for NoSQL database functionality, and Apache Spark for faster, in-memory data processing. These tools collectively form a robust ecosystem that enables organizations to tackle big data challenges efficiently, making Hadoop a cornerstone in the world of data analytics and processing.


NoSQL: NoSQL, short for "not only SQL," represents a family of database management systems designed to handle large and unstructured data. Unlike traditional relational databases, NoSQL databases offer flexibility, scalability, and agility. They are particularly well-suited for applications involving social media, e-commerce, and real-time analytics. Prominent NoSQL databases include Hbase for columnar storage used extensively in Hadoop Ecosystem.


Spark: Apache Spark is an open-source, lightning-fast data processing framework designed for big data analytics. It offers in-memory processing, which significantly accelerates data analysis and machine learning tasks. Spark supports various programming languages, including Java, Scala, and Python, making it accessible to a wide range of developers.  With its ability to process both batch and streaming data, Spark has become a preferred choice for organizations seeking high-performance data analytics and machine learning capabilities, outpacing traditional MapReduce-based solutions for many use cases.

Who this course is for:

  • People looking to advance their career in Data Engineering, Big Data, Hadoop, Spark
  • Hadoop python spark scala hive pig oozie sqoop flume kafka

Report abuse