Elasticsearch Illuminated: A Practical Guide

Name: Elasticsearch Illuminated: A Practical Guide
Rating: 4.5 (10 reviews)

Utilizing ELK Stack in Building Complex Architectures

Highest Rated

Created byFaizan Tayyab

Last updated 11/2022

English

What you'll learn

Understand the basics of Elasticsearch
Understand the ELK stack (Open Source)
Build Applications utilizing Elasticsearch
Elasticsearch as part of complex archietectures

Course content

14 sections • 203 lectures • 6h 20m total length

Introduction1:34
Intro to the course & Instructor
Scope0:19
Scope of the course
ELK Architecture1:21
What is ELK stack
Elasticsearch0:50
Introduction to Elasticsearch
Why Elasticsearch1:19
Why use elasticsearch
Elasticsearch common use cases1:34
Common use cases for Elasticsearch
Configuration over code2:38
Explain configuration over code and use of configuration in ELK stack
System Requirements0:48
Important system requirements

Download & Install Elasticsearch I1:23
Learn how to download Elasticsearch from the elastic.co website
Download & Install Elasticsearch II1:08
Extracting files to run Elasticsearch
[Important] Choosing short paths (Windows)0:22
Important for windows platform users
[Important] Firewalls and security settings (Windows)0:32
Important Security Restrictions on Windows
Running Elasticsearch (Local)2:59
Learn how to locally run Elasticsearch on Windows
Running Elasticsearch (non-Windows)0:22
Instruction to run Elasticsearch
[Optional] Running Elasticsearch (Docker)1:42
How to run elasticsearch on docker
Important (Certificate Warning, Docker version)0:36
default certificate warning
[Optional] Verify Elasticsearch is up (Docker)2:40
Verify Elasticsearch is running inside a container on local machine
Elasticsearch Default Configuration (Local Machine)4:02
Default configuration of Elasticsearch
Important commands0:14
Commands for setting password and getting enrollment tokens
Verify Elasticsearch is up and running (Local)1:24
Verify Local installation of Elasticsearch
Important Information about the Certificate Warning0:45
[Optional] Sorting the Certificate Warning1:19
Solving Certificate warning on Chrome
Elasticsearch Installation Review0:57
Review of the installation steps
Download & Install Kibana I0:36
Download Kibana
Download & Install Kinbana II [Enrollment]3:38
Setup Kibana and run in browser
[Optional] Containerized Kibana0:16
Running Kibana as containerized App
Downloading Logstash0:54
How to download Logstash
Examining Logstash Installation2:13
Examine Logstash Installation
Important About next lesson0:29
Important note about logstash
Verify Logstash (Simple Pipeline)1:46
A simple pipeline example via logstash
Downloading FileBeats (Beat)0:35
Download FileBeats from elastic.co
FileBeats Installation0:48
learn about Installing Filebeats
Important about next lesson (Data Shippers)0:14
Note about Beats.
Verify FileBeat installation (Shipping Data via FileBeat)5:38
Demo on how to use Filebeats to ingest from a log file into Elasticsearch
Section Review1:09
Review of the installation section
[Troubleshooting] Kibana -> Elasticsearch connection Issue1:55
Kibana to elasticsearch connection issues
[Troubleshooting] Problem Starting Zookeeper0:18
Issue with starting zookeeper

Elasticsearch Modes (Single vs Cluster vs SAAS)3:11
Distributed Systems (Elasticsearch)
Elasticsearch Rest APIs0:48
Learn how Elasticsearch is setup on top of Leucene and the API it provides
Restful API1:37
Explain Rest API
Understanding JSON I2:29
Learn basics of JSON
Understanding JSON II3:55
Understanding JSON and creating a JSON document
Optional [Starting Elasticsearch & Kibana]0:56
Starting Elasticsearch and Kibana if not already started. This is optional lesson.
Client for Rest API (Kibana Dev Console)1:17
Using Kibana Dev tools for queries
Inside Elasticsearch Cluster I6:22
Explain Cluster, nodes, index, shards, documents and fields
Inside Elasticsearch Cluster II4:26
Inside the cluster, Index, Cluster, Nodes, shards and fields
Inverted Index1:41
Description on Inverted Index
Review: Partitioning, Shards and Index6:50
Detailed explaination on partitioning, indices and shards
Cluster API3:58
Using the Cluster API to get cluster informaton
Segments1:29
Some information about Segments and what they are
Analyzers3:31
Introduction to Analyzers and what role they play in indexing and text analysis
Mappings1:54
What are mappings including dynamic and explicit mappings
Relevance4:10
TD/IDF, Score and Relevance

Data for this and next section [Important]9:15
Important for next few lessons
Creating a document I2:34
Using POST requests and learning about issues with duplicates
Creating a document II [Avoid Duplicates]1:41
Learn to avoid duplicates when inserting documents
Creating documents III [PUT]0:44
Using PUT to insert documents
Geting a document1:22
Getting a document
Bulk Create1:11
Bulk Insert
Search Query I4:48
Searching for documents
Documents Relevance (Similarity)5:07
Undestand document scoring
Controlling Relevancy [Boosting Parameter]1:25
Learn to use Boosting Parameters
Controlling Relevancy [Boosting Query]2:00
Boosting query to control relevancy and how scoring algorithm works
Term Query [Recap]1:27
Understand term query
Search Query II [Bool Query]3:11
Bool queries and Filters
Search Query III [Filters]1:39
Using filters with bool query
Fuzzy Query2:04
Learn to use Fuzzy Queries
Keyword vs Text6:07
Text vs Keyword Type
Updating a document I1:14
Document update.
Updating a document II3:16
Updating fields and documents
Deleting a Document1:04
Deleting a document
Creating an Index3:11
Learn about explicit mapping and index creation
Reindexing Documents0:48
Learn about reindexing documents
Node & NPM [Important]1:11
Important Information
Using ElasticDump [Import to ES]1:36
Learn how to use ElasticDump to import csv file in index
Using ElasticDump [Export from ES]1:19
Exporting from ES index to a File
Loading CSV data into Elasticsearch via Kibana0:09
Reference to other section
JSON & NDJSON1:29
Learn about NDJSON format
Loading JSON data into Elasticsearch via Kibana Upload3:37
How to load (index) JSON data onto Elasticsearch and dealing with standard JSON
Kibana Data View [customers index]1:23
A short introduction to data views
Loading Apache Log Data [GROK]3:20
Understand GROK
Understanding GROK with example4:42
Example of GROK Pattern
Data Ingestion Architecture2:05
Learn about simple architecture of how to insert documents into Elasticsearch index using the bulk api and python client
Data Ingestion Bulk insert Demo [Python]5:50
Bulk Insert demo
Using SQL with Elasticsearch1:11
Learn to use SQL with ElasticSearch

Analyzers I [Built-in Analyzers]2:45
How to work with built-in analyzers
Analyzers II [Custom Analyzer]2:13
Building custom analyzers
Ngram Tokenizer1:55
Learn how Ngram tokenizer works
Edge Ngrams1:20
Learn about Edge Ngrams and why they are so useful
Runtime Fields1:26
Learn about runtime fields
Creating a policy [ILM]4:02
ILM (Creating a policy)
Apply a policy [ILM]2:01
Applying a policy
Aggregations4:52
Learn to work with Aggregations

Data Shippers0:44
Learn what are data shippers and why Beats are called lightweight data shippers
Beats and Logstash1:21
beats and logstash, when to use what?
Beats Configuration0:38
Learn about configuring beats
Uptime Monitoring using Heartbeat0:27
What is heartbeat and what it is used for
Download & Installation0:22
Download and Install heartbeat
Configuration1:13
Configuring Local installation of heartbeat
Testing Configuration0:48
Testing Heartbeat Configuration for errors
Enabling Heartbeat Dashboards1:38
Installing the Heartbeat Dashboard
Running Heatbeat1:51
Running Heartbeat and visualizing data using Kibana
Elasticsearch and Kibana Status0:11
Running File Beat Review5:46
Filebeat review. How to configure and run filebeat
Data Streams1:06
Understand what are data streams

Why Data Modelling1:46
Intro to data modelling and why we need data modelling. What benefits it offers
Data Modelling2:48
Data Modelling Intro including discussion on steps to perform NoSQL data modelling.
Data Modelling (SQL vs NoSQL)1:58
Comparision of relational and non relational way of modelling data
Elasticsearch Data Modelling Patterns1:30
Data Modelling Patterns
Importing Data for this section0:38
Important instructions
Case Study2:53
Simple case study presented
Relationship Patterns: Application Joins1:24
App Joins : How they work
Relationship Patterns: Application Joins Demo6:36
Demo of application join in action via Console Dev Tools
Loading Data for Inner Objects0:34
Loading data for Inner Objects
Inner Objects1:19
How inner objects work
Problems of Inner Objects Demo4:40
Inner Object Demo in Console
Inner Objects Flattening3:20
More explanation of flattening
Loading Data for Nested Objects0:17
Loading data for nested Objects example
Relationship Patterns: Nested Objects1:11
Nested Types
Relationship Patterns: Nested Objects Demo1:42
Nested Objects demo and queries
Relationship Patterns: Parent Child relationships Introduction0:46
Define new Index. Put documents and query data
Relationship Patterns: Parent Child relationship Intro2:02
Parent Child relationships introduction
Relationship Patterns: Parent Child relationship Demo6:09
Demo of parent child relationship
Relationship Patterns: Denormalisation1:11
What is Denormalisation?
Relationship Patterns: Denormalisation Code0:17
Queries for Denormalization
Relationship Patterns: Denormalisation Demo2:20
Denormalization Demo in Console
Relationship Patterns: Inversion Pattern1:44
How inversion pattern works
Factors to consider (Embedding vs Reference)1:02
Data Modelling considerations

About this section0:22
Elasticsearch as Database2:58
What we will be looking at in this section
Example architecture: Elasticsearch as a DataStore2:09
Architecture example
Load plant Data [CSV]2:37
Loading plant data vua kibana
Checking Plant Index mappings1:07
Checking and verify the Plant index mapping are as expected. We changed the mappings in the last lesson while loading plant data via Kibana
Code Editors0:05
Setting up and Running the Project0:52
Learn how to setup and run NodeJS and Javascript application utilizing Elasticsearch
Application Demo2:05
Javascript Application built
Dependencies (packages)0:27
Required Packages
Making a connection0:52
making a connection to the elasticsearch cluster running on the local machine
Server side code (NodeJS-ES integration)1:35
Server side Code Explaination (NodeJS)
Front end (Pure Javascript Application)1:11
Front end Javascript review
Application Review0:36
Final Review of the section
Binary Data0:48
What is binary data and how ES supports binary data
What we will build2:34
Second applicaiton we will be building in this section
Transforming raw data1:35
Creating Index and specifying custom mappings1:03
Creation of Indec with Binary field
Important (installing all the required packages)0:13
Loading data via custom application (Node JS)2:16
Code to load data into index
Verifying Stored data0:46
verify data is in the index
Server side processing1:31
Server side component
Client side Application1:04
Client side application (Map with Data)
Application Demo0:43
Application demo (Map)

Requirements

Programming experience in an object oriented Language
Ability to use command Line
Windows OS

Description

This course has a broader scope and covers not just the ELK stack but also integration of Elasticsearch with other technologies that are commonly used in the industry to build efficient, scalable and powerful architectures. The course starts with the basics and guides the student on the ELK stack, provides useful information on how to install various components of the ELK stack. Certain parts of the lessons are intentially repeated to reinforce the ideas and help with improved learning. This course can be taken by both beginners and experienced Professionals to learn the technology or refresh previous knowledge. The course covers the latest version 8 of the ELK stack however do note the minor versions are being released almost every 2 months therefore, the version on the course may be slightly behind but all that is discussed in the course should work with the latest minor version of ELK stack.

It is highly recommended that student has background in software development or possesses programming skills. For those who do not, they can follow along or just watch the lessons. The course has been desgined to enable student to learn the exciting stack and help them understand how they can adopt ELK stack in their current & future projects.

Who this course is for:

Software developers
Software Engineers
Programmers
Architects

Elasticsearch Illuminated: A Practical Guide

What you'll learn

Explore related topics

Course content

Introduction8 lectures • 10min

Installations29 lectures • 41min

Elasticsearch Architecture16 lectures • 49min

Elasticsearch CRUD Operations32 lectures • 1hr 22min

Advanced Elasticsearch8 lectures • 21min

Kibana Visualization and Analytics4 lectures • 7min

Data Shippers12 lectures • 16min

Logstash5 lectures • 7min

NoSQL Data Modelling23 lectures • 48min

Elasticsearch As Data Store (App Development)23 lectures • 29min

Requirements

Description

Who this course is for: