Hadoop emerged in response to the proliferation of masses and masses of data collected by organizations, offering a strong solution to store, process, and analyze what has commonly become known as Big Data. It comprises a comprehensive stack of components designed to enable these tasks on a distributed scale, across multiple servers and thousands of machines.
This course introduces you to the powerful system synonymous with Big Data, demonstrating how to create an instance and leverage Hadoop ecosystem's many components to store, process, manage, and query massive data sets with confidence.
The video course opens with an introduction to the world of Hadoop, where we discuss Nodes, Data Sets, and operations such as map and reduce. The second section deals HDFS, Hadoop's file-system used to store data. Further on, you’ll discover the differences between jobs and tasks, and get to know about the Hadoop UI. After this, we turn our attention to storing data in HDFS and Data Transformations. Lastly, we will learn how to implement an algorithm in Hadoop map-reduce way and analyze the overall performance.
About The Author
A K M Zahiduzzaman is a software engineer with NewsCred Dhaka. He is a software developer and technology enthusiast. He was a Ruby on Rails developer, but now working on NodeJS and angularJS and python.He is also working with a much wider vision as a technology company. The next goal is introducing SOA within the current applications to scale development via microservices.
Zahiduzzaman has a lot of experience with Spark and is passionate about it. He is also a guitarist and has a band too. He was also a speaker for an international event in Dhaka. He is very enthusiastic and love to share his knowledge.
In this video we’ll learn how to install Hadoop on our local system
The important part of selecting the Hadoop framework for your own solution is to understand why it is a good fit for your application.
Understand the difference between the two nodes in HDFS; Datanode and Namenode
The new term Map-Reduce… what does it mean and how does it solve a problem?
When jumping to parallel programming from serial programming, it is always hard to plan the computation.
Copy data to/from HDFS.
Using the HDFS commands in the shell.
How do we access the HDFS files from a java program
How to see the process flow and progress of a Hadoop job.
Run Hadoop jobs.
In this video, we are going to look at how the map and reduce gets executed
Understand the dataset provided by the grouplens.org
Prepare the data to be fit for our algorithm
Devise a simple algorithm for recommendation
Implement the map-reduce for the transformation of the movie -> genre context
Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.
With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.
From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.
Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.