This course is a step-by-step guide to using Elasticsearch in combination with the rest of the ELK stack (now called Elastic Stack) to ship, parse, store, and analyze logs.
You’ll start this course by getting an understanding of what ElasticSearch is, what it’s used for, and why it’s important. Then, you’ll be introduced to the new features in ElasticSearch 5.0. We’ll go through each of the fundamental concepts of ElasticSearch such as queries, indices, and aggregation.
You’ll find out how to add more power to your searches using filters, ranges, and more. You’ll also see how ElasticSearch can be used with the other components of the Elastic Stack such as LogStash, Kibana, and Beats. Finally, we’ll take a walk through getting ElasticSearch up and running on the popular logstash Apache web log analysis.
Aside from learning how to add more power to your searches with filters, ranges, and more, you'll also see how ElasticSearch can be used with the other components of the Elastic Stack such as LogStash, Kibana and Beats. Finally, we’ll build, test and run our first Logstash pipeline to analyze Apache web logs. This step combines all the understanding of ElasticSearch, Logstash, Kibana and the lightweight data shipper FileBeat that we acquired from previous sections.
By the end of this course, you will have a firm understanding of all the fundamentals of ElasticSearch 5.0, along with knowledge of practical real world usage.
About The Author
Ethan Anthony is a San Francisco based Data Scientist who specializes in distributed data centric technologies. He is also the Founder of XResults, where the vision is to harness the power of data to innovate and deliver intuitive customer facing solutions, largely to non-technical professionals. Ethan has over 10 combined years of experience in cloud based technologies such as Amazon webservices and OpenStack, as well as the data centric technologies of Hadoop, Mahout, Spark and ElasticSearch. He began using ElasticSearch in 2011 and has since delivered solutions based on the Elastic Stack to a broad range of clientele. Ethan has also consulted worldwide, speaks fluent Mandarin Chinese and is insanely curious about human cognition, as related to cognitive dissonance.
To get a feel for the course we do an end-to-end overview of what will be covered.
Gaining a holistic view of a new technology is the first necessary step to learning how it works. ElasticSearch is introduced with accompanying use cases.
Setting up a new technology is often a challenging affair. By walking through the simple process of installing ElasticSearch, developers can quickly move along the learning process.
Before diving into any new technology, it is all too important to understand what the subject technology was designed for and the best use cases. Peering into the objectives of ElasticSearch solves this.
Version 5.0 of ElasticSearch has some key changes. Highlighting these changes help developers to get a better understanding of what's new.
As is the case with any new technology, developers seek compelling use cases for implementing ElasticSearch. ElasticSearch answers the call by being a fairly straight forward, developer friendly analytics engine.
The indices are easily the cornerstone of ElasticSearch. As such, understanding indices and how they work is key.
Documents hold data in ElasticSearch. Understanding the workings of documents put one on the path to better understanding ElasticSearch.
The concept of a cluster can be broad and sometimes confusing. Understanding an ElasticSearch cluster is an important step in the learning process.
Distributed technology is extremely challenging to understand. Understanding how to set shards and replicas in ElasticSearch is therefore a necessary first step.
Index and Mapping set the stage for data search and analysis. Knowledge of how each work is important for effective ElasticSearch usage.
Document addition and deletion in ElasticSearch controls the flow of data. The ability to add and delete documents in ElasticSearch is necessary.
Since adding documents is among the most commonly performed tasks in ElasticSearch, there needs to be a way to add multiple documents simultaneously. The bulk API solves this problem.
Interfacing with technology from external systems can be challenging and often requires a high degree of expertise. The REST API in ElasticSearch solves this problem.
Using REST API requires knowledge of how to run desired queries. Hands-on experience makes this possible.
One common point of confusion in REST technology is differentiating between PUT and POST. Gaining a clear understanding of PUT versus POST in updates is therefore key.
Accessing the power of ElasticSearch necessitates understanding of its query language, DSL. A breakdown of DSL and how it works is essential.
Moving beyond the basics in DSL can be challenging. Going beyond the basic to take a deeper look into DSL helps.
Understanding the type of queries required to gain optimal results is necessary in ElasticSearch. Knowledge of term queries and boosting helps to optimize query results.
Sometimes you will want to search a range of values. Range queries are the solution to this challenge.
At times you will need to know when a given field exists. This is when you should turn to exist queries.
Generating analytics can be a challenging task. Built-in aggregation based analytics in ElasticSearch take the pain out of analytics.
The process of running aggregation based analytics in ElasticSearch can get confusing. Experience is the best teacher.
ElasticSearch alone doesn't provide security, cluster management, log analysis and so on. Thus, ElasticStack was created.
Data aggregation is the first of many steps in the analysis process. Kibana facilitates data visualization and acts as a cluster management interface.
Log analysis is an involved process that moves unstructured log data into the ElasticSearch cluster. Logstash was created for this very reason.
Prior to ElasticSearch 5.0, security, monitoring, alerting, reporting, graph and so on. were all separate components. Version 5.0 combined these to form X-Pack.
The complexity of moving data from external systems to ElasticSearch presents many challenges. Beats was created to simplify the process of moving data into ElasticSearch.
Log analysis is a multi-step process that requires attention to detail. Clear understanding of the process is essential.
It's not enough to know the steps to the log analysis process. Practical experience is also necessary for true understanding.
In the world of data management, sorting is an absolute must have feature. Learning to sort in ElasticSearch can greatly improve search results
The ability to query Geo data highly ranks in modern use cases. ElasticSearch simplifies the process of geo searching.
User generated queries for text search can be filled with colloquialism, abbreviations etc. Synonyms in ElasticSearch aim to help minimize this challenge.
Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.
With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.
From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.
Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.