Master ElasticSearch on Hadoop Ecosystem with Data Pipelines
4.2 (18 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
176 students enrolled
Wishlisted Wishlist

Please confirm that you want to add Master ElasticSearch on Hadoop Ecosystem with Data Pipelines to your Wishlist.

Add to Wishlist

Master ElasticSearch on Hadoop Ecosystem with Data Pipelines

Learn A-Z of Integrating ElasticSearch & Hadoop with Hands-on Examples of Building Hive, PIG, LogStash,MR Data Pipelines
4.2 (18 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
176 students enrolled
Created by Neeraj Malhotra
Last updated 3/2017
English
Current price: $10 Original price: $180 Discount: 94% off
5 hours left at this price!
30-Day Money-Back Guarantee
Includes:
  • 6 hours on-demand video
  • 1 Article
  • 1 Supplemental Resource
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
What Will I Learn?
  • Advance your career in Big data by learning how to integrate ElasticSearch on Hadoop Ecosystem and create real world data pipelines for your big data applications.
  • Develop sound understanding of data ingestion, integration across systems, full text search & data analytics.
  • Develop in demand skills for building Data Pipelines using Apache Hive, Pig, MapReduce (Java), & LogStash to index data into ElasticSearch clusters for quick data analytics and text searching.
  • Advance your career by applying for an awesome job with great pay.
  • Develop awesome data visualizations using Kibana.
View Curriculum
Requirements
  • Access to a computer with Mac/Windows/Linux operating system installed. Windows/PC users will need VMBox (Virtual Machine) installed on machine (for exercises only).
  • Internet connection (to watch course videos and download necessary tools for the course)
  • Passion about Big Data technologies.
Description

Master Integrating ElasticSearch in Hadoop Ecosystem with hands on examples of building real world Data Pipelines using Apache Hive, PIG, MapReduce and LogStash.

The only course on Internet covering integration of ElasticSearch with Hadoop and creating various real world applications.

No other course can cover so much ground as we will do in this one.

In this course, you will learn step by step about ElasticSearch and how to integrate with all major components in Hadoop ecosystem. With real world examples, you will learn how to build various Data Pipelines such as:

Ingestion Flows (to ElasticSearch)

  • Apache Hive to ElasticSearch
  • Apache PIG to ElasticSearch
  • MapReduce to ElasticSearch
  • LogStash to ElasticSearch

Egression Flows (from ElasticSearch)

  • ElasticSearch to Apache Hive
  • ElasticSearch to Apache PIG
  • ElasticSearch to MapReduce
  • ElasticSearch to LogStash

Data Visualization

  • Visualize indexed data using Powerful Kibana and Real time Dashboards

Production Cluster Monitor tool

  • Cluster Health monitoring at Index, Shard, Node levels
  • Parsing ElasticSearch Cluster statistics using Linux utilities
  • Setting up wait-for-trigger mechanism and much more

You will also learn about awesome search capabilities offered by ElasticSearch and how to query vast index of data in real time. This will be really fun!!!

We will cover lots of basics to build foundation required to understand ElasticSearch. You will also learn about behind the scenes on how a search engine and specifically ElasticSearch works in a single or multiple node cluster.

You will also get step by step instructions for installing all required tools and components on your machine in order to run  all examples provided in this course. Each video will explain entire process in detail and easy to understand manner.

You will get access to working code for you to play with it and expand on it. All code examples are working and will be demonstrated in video lessons.

Who is the target audience?
  • This will be an excellent course for anyone who wants to learn about Big Data technologies and how to use them together in order to create amazing Big Data applications.
  • Big Data Developers, Architects, Data Scientists, Data Analysts and Students (with zero experience)
Students Who Viewed This Course Also Viewed
Curriculum For This Course
145 Lectures
05:56:04
+
Building Foundation
13 Lectures 14:09


Course Resources
00:09

What is a Search Engine
01:09

Inside a Search Engine
01:24

What is MetaData
01:56

ElasticSearch in a Nutshell
01:21

How ElasticSearch offers Scalability
01:41

ElasticsSearch provides High Availability
01:31

Multi-Tenancy Out of the Box
00:57

Full Text Search inside ElasticSearch
00:52

Real Time Analytics with ElasticSearch
00:49

Chapter Summary
00:42

Quiz
1 question
+
Setup a Working Environment
17 Lectures 40:08
Introduction
00:51


[Hands-on] - Lets start with installing Homebrew
01:52

[Hands-on] - Installing wget
01:44

[Hands-on] - Check Java Version installed
00:53

[Hands-on] - Check and Enable SSH
03:39

[Hands-on] - Getting Latest Apache Hadoop
04:09


[Hands-on] - Configuring Apache Hadoop Part - 2
02:48

[Hands-on] - Configuring Apache Hadoop Part - 3
05:48


[Hands-on] - Getting ElasticSearch
01:32

[Hands-on] - Configuring ElasticSearch
05:03

[Hands-on] - Installing ElasticSearch Plugins: <Head Plugin>
01:26

[Hands-on] - Installing ElasticSearch Plugins: <Marvel Plugin>
01:01

[Hands-on] - Starting ElasticSearch Daemon and Checking out Plugins
02:40

Chapter Summary
00:29

Quiz
2 questions
+
Building Blocks of ElasticSearch
11 Lectures 14:23
Introduction
00:14

RDBMS vs ElasticSearch
01:39

How a document (data record) looks likes inside ElasticSearch
01:19

Marvels of Inverted Index
02:02

Shards - low level storage units in ElasticSearch
00:41

An ElasticSearch Node
01:54

ElasticSearch Cluster
00:54

Monitoring ElasticSearch Cluster's Health
02:44

How to Scale an ElasticSearch Cluster
01:43

RestAPIs in ElasticSearch
00:41

Chapter Summary
00:32

Quiz
1 question
+
Operations in ElasticSearch
16 Lectures 45:12
Introduction
00:25


Types of Operation in ElasticSearch
00:44

Inside Operations: <Write> & <Delete>
02:25

[Hands-on] - Lets try a Write Operation
04:50

Inside Operations: <Read>
02:30

[Hands-on] - Lets try a Read Operation Part-1
04:55

[Hands-on] - Lets try a Read Operation Part-2
01:43

Inside Operation: <Update>
03:29

[Hands-on] - Lets try an Update Operation
03:43

[Hands-on] - Lets try a Delete Operation
03:19

Concept of Mapping in ElasticSearch
01:49

[Hands-on] - How to add Mappings on Indexed data
06:29

Data consistency using Templates
01:10

[Hands-on] - Using Templates
04:53

Chapter Summary
00:58

Quiz
1 question
+
Queries in ElasticSearch
9 Lectures 31:02
Introduction
01:19

Types of Search Queries
00:51

[Hands-on] - Creating Dataset for Search
06:17

[Hands-on] - Using QueryString Part-1: Select All
06:46

[Hands-on] - Using QueryString Part-2: Filter Specific Fields
03:30

Word of Caution with QueryStrings
01:26

[Hands-on] - Using DSL Queries Part-1
05:49

[Hands-on] - Using DSL Queries Part-2
04:30

Chapter Summary
00:34
+
Data Pipelines
1 Lecture 00:49
+
Data Pipeline#1 - Apache Hive to ElasticSearch
10 Lectures 38:40
Setting Objectives
00:22

[Hands-on] - Installing Apache Hive
01:52

[Hands-on] - Configuring Apache Hive Part-1
02:48

[Hands-on] - Configuring Apache Hive Part-2
03:59

Getting ElasticSearch Connector JAR
03:40

Where to get free datasets for exercises
01:03

Understanding Our Dataset
01:13

[Hands-on] - Creating a Data Flow from Hive to ElasticSearch Index Part-1
09:56

[Hands-on] - Creating a Data Flow from Hive to ElasticSearch Part-2
10:17

[Hands-on] - Looking at ingested data inside ElasticSearch Cluster
03:30

Quiz
2 questions
+
Data Pipeline#2 - ElasticSearch to Apache Hive
5 Lectures 12:07
Introduction
00:17

Setting Objectives
00:27

[Hands-on] - Indexing Data inside ElasticSearch using Bulk API
03:36

[Hands-on] - Creating Data Flow from ElasticSearch Index to Hive table
07:05

Chapter Summary
00:42

Quiz
2 questions
+
Data Pipeline#3 - Apache PIG to ElasticSearch
9 Lectures 21:24
Introduction
00:21

Setting Objectives
00:29

Basics about Apache PIG
01:09

[Hands-on] - Installing Apache PIG
02:45

[Hands-on] - Configuring Apache PIG
02:01

Lets up a level by introducing Apache Hive into picture
00:57

[Hands-on] - Getting Dataset for the exercise
00:54

[Hands-on] - Creating data flow from Apache PIG to ElasticSearch Part-1
02:45

[Hands-on] - Creating data flow from Apache PIG to ElasticSearch Part-2
10:03

Quiz
1 question
+
Data Pipeline#4 - ElasticSearch to Apache PIG
4 Lectures 08:35
Introduction
00:17

[Hands-on] - Getting the Dataset
00:35

[Hands-on] - Creating a Data Flow from ElasticSearch to Apache PIG to HDFS
06:50

Chapter Summary
00:53
7 More Sections
About the Instructor
Neeraj Malhotra
4.2 Average rating
18 Reviews
176 Students
1 Course
Senior Big Data Architect

My name is Neeraj Malhotra and I am super excited for Big Data world. I am honored that you are here & reading this!!!

Background

I have over 13 years of experience in IT consulting & software development. I have worked for all major Telecom providers in North America on providing solutions for millions of their customers and managing huge data stores. Having an instinct for data, I volunteered for Big Data projects within the organization and lead various giant projects with exciting journey of challenges, learnings and achievements.

Why Udemy

I am planning to share my knowledge through Udemy courses with others and help them explore this exciting world of software development especially Big Data technologies. 

Philosophy

I strongly believe that knowledge spreads with sharing and learnings become more concrete with teaching about it to others.