
See why traditional systems can't meet today’s data needs and how Hadoop differs from relational databases. Learn Hadoop’s components—SBF or Hadoop distributed file system, Map Reduce, and Yarn—for cluster management.
Explore why big data demands distributed computing, how Hadoop enables scalable storage and processing, and the key role of associated technologies in handling Facebook, Google, and similar data volumes.
Compare monolithic and distributed systems, showing how a cluster of commodity hardware enables linear scaling by adding nodes, with coordination software managing partitioning, fault tolerance, and parallel tasks.
Explore the fundamentals of Hadoop, including the Hadoop distributed file system and MapReduce, and see how jobs run across a cluster with Yarn.
Compare Hadoop and RDBMS, highlighting unstructured data, schema development during processing, and absence of ACID transactions. Explain cap theorem and Hadoop's horizontal scaling, highlighting higher throughput yet slower granular queries.
Explore why distributed computing surpasses a single computer setup and identify the roles of players in a Hadoop ecosystem, with a big-picture view of related technologies.
Explore why Hadoop enables big data analysis and how HDInsight addresses its challenges, with a demo processing data through Hive in the cluster and storing results.
Explore why big data projects struggle despite Hadoop's promise, focusing on upfront hardware costs, scalability limits, and the data talent shortage in on-premises deployments.
HDInsight makes Hadoop easy as a fully managed cloud service for Hadoop clusters with elastic, scalable computing and SLA-backed support, while eliminating upfront hardware costs and enabling pay-as-you-go deployment.
Important Note:
Please note that this is NOT a full course but a single module of the full-length course, and intended to cover very basic fundamental concepts for absolute beginners so that they can speed up with Azure Synapse SQL Data Warehouse course.
This module is NOT GOOD for you if:
You are already experienced in this technology
You are looking for an intermediate or advance concepts
You are looking for practical examples or demo
This module is GOOD for you if:
You want to understand the basic fundamental concepts of this technology.
This is a free module to help others. If you are not in the intended audience, I request you to please feel free to unenroll.
Where I can find a full-length course?
Please look at the bonus lecture in the end.
What will students learn in this course?
Hadoop basic concepts (Crash course to speed up with Azure HDInsight)
Level
Beginners
Language
English
If you are not comfortable in English, please do not take course, captions are not good enough to understand course.
Target Students
Database and BI developers
College students
Anyone who wants to start learning Big Data
Prerequisites
Basic Database concepts
Course In Detail
Data Warehouse Crash Course
In this module, you will learn, what was shortcomings of our traditional Monolithic systems.
you will learn how Distributed system is different from Monolithic systems
you will learn about Hadoop fundamental understanding and how it is different from RDBMS
you will learn 3 main building blocks or components of Hadoop, like the HDFS or Hadoop Distributed File System, the MapReduce programming model for processing and the resource negotiator YARN for cluster management.
[Tags]
Microsoft SQL Server, Azure SQL Server, Azure SQL Data Warehouse, Data Factory, Data Lake, Azure Storage, Azure Synapse Analytics Service, PolyBase, Azure monitoring, Azure Security, Data Warehouse, SSIS