Complete Guide to Elasticsearch

Learn what sharding is, and how it helps scale an Elasticsearch cluster, both in terms of data volume and query throughput.

Let's be sure that you understand the most important concepts of sharding.

Learn what replication is, and how it helps ensure high availability as well as increased query throughput.

Let's be sure that you understand the most important concepts of replication.

Learn how to form a multi-node cluster for development purposes.

Elasticsearch nodes have roles, which can be configured for each node. Get an overview of which roles are available, and when you should use them.

Let's take a quick moment to wrap up this section of the course.

Before adding some data into our cluster, let's see how to both create and delete indices. When creating an index, we will supply two settings for specifying the number of shards and replica shards for the index.

In this lecture, you will learn how to index documents in two ways; by having Elasticsearch generate an ID for documents automatically, and by specifying an ID ourselves.

Learn how to retrieve a document based on its ID.

Learn how to update an existing field's value, and how to add a new field to a document.

Learn how to update documents with scripts. Scripts enable us to update documents dynamically, such as by performing calculations on field values. You will also see how to include parameters within scripts. Lastly, you will learn how to ignore a given document during an update, how to conditionally update a document, and how to delete a document within a script.

In this lecture, we will cover so-called upserts. An upsert is a way to index a document or update an existing one, depending on whether or not a given document already exists.

Apart from updating documents, we can also replace them altogether. You will learn how to do just that in this lecture.

The last document action you need to learn, is to delete documents. Fortunately, this is extremely easy due to Elasticsearch's REST API.

How did Elasticsearch find the documents that we have indexed in the previous lectures? And how did it know where to store the documents that we indexed? That's what we will be looking at in this lecture.

Now that you know what routing is, let's take a closer look at how Elasticsearch reads data. This involves a coordinating node and Adaptive Replica Selection (ARS).

Now that you know how Elasticsearch reads data, let's talk about how it writes data. That's a more complicated subject involving something called primary terms, sequence numbers, as well as global and local checkpoints. We will also cover the basics of how data is replicated to replica shards.

Elasticsearch uses a primitive kind of document versioning. Let's take a look at what that is all about.

Documents might be overwritten inadvertently due to a web server running multiple threads in parallel, for instance. How do we deal with this problem? We first look at how we used to handle this, before diving into how primary terms and sequence numbers can be used to overcome this challenge.

Let's see how we can update multiple documents with a single query, similar to an UPDATE WHERE query for relational databases. We will also go through the inner workings of the query, including how it handles errors and performs optimistic concurrency control.

Now that you know how to update multiple documents with one query, let's do the same thing, just in regards to deleting documents.

Learn how to process multiple (potentially thousands or millions) actions within one query. This can be used to index, update, or delete many documents at the same time.

It's time to import some test data. Learn how to import data by using the Bulk API together with cURL on the command line.

Let's take a quick moment to wrap up this section of the course.

Quick introduction to what is covered in this section of the course.

Introduction to the concept of text analysis.

Learn how to use the Analyze API to test built-in and custom analyzers. It can also be used to test character filters, tokenizers, and token filters.

Learn how text values are analyzed and stored within a data structure called an inverted index.

Introduction to the concept of mapping.

An overview of the most important data types in Elasticsearch, with a focus on the "object," "nested," and "keyword" data types.

Let's take a closer look at how the "keyword" data type works and learn why it is used for exact matches, aggregations, and sorting.

Learn what type coercion is all about and how Elasticsearch can sometimes convert from one data type to another.

How can we index array values into Elasticsearch? How do arrays work, and what are their limitations? Learn the answers to those questions in this lecture.

Time to add our first mapping to an index.

Learn how to retrieve mappings for an index, and also specific field mappings.

Mapping queries can be simplified when using a dot notation syntax for nested fields. Learn how to do that in this lecture.

Sometimes you might want to add field mappings to an existing index. Learn how to do that in this lecture.

Let's take a look at how Elasticsearch handles dates. In particular, which date formats are supported, how timezones work, how Elasticsearch stores dates internally, and how dates are searched by search queries.

What happens if we leave out a field when indexing a document? And does adding a field mapping make a field required? Those are some of the questions that we will answer in this lecture.

In this lecture, we take a look at the most important mapping parameters, including "doc_values," "norms," "index," and more.

In this lecture you will learn how to update existing field mappings. There are quite a few limitations to updating mappings in Elasticsearch, so we will cover those as well.

Let's see how we can reindex documents into a new index. We will take a look at how the Reindex API works and cover a number of common use cases with examples.

Instead of renaming fields, we can make use of field aliases. You will see how to add and use field aliases in this lecture.

An Elasticsearch field can actually be mapped in multiple ways. This is very useful for querying field values in different ways.

Index templates can be used to specify field mappings and index settings. These can then automatically be applied to new indices based on index patterns. We will take a look at common usages of index templates, among others.

In this lecture, we will look at how the Elastic Common Schema (ECS) defines a large number of common fields and their data types. We will also take a look at why ECS was created, and when it is useful.

Time to get into dynamic mapping now. We will take a look at the rules that Elasticsearch uses when dynamically mapping fields, along with a couple of examples.

Now that you know of both explicit and dynamic mapping, what about combining them? As you will see, you don't necessarily have to choose either explicit or dynamic mapping.

Let's take a look at how we can configure dynamic mapping. You will see how to disable it, how to set it to "strict," and how inheritance is supported. Apart from that, we will also look at numeric detection and date detection.

Dynamic mapping can be configured with dynamic templates. These templates define certain conditions, and when they are matched, the specified mapping will be applied.

Before moving on to analysis and analyzers, I have a few recommendations in regards to mapping.

Stemming and stop words are two common things to handle during text analysis. We will look at how words can be stemmed to their root form as a way to improve search results. We will also cover the concept of removing stop words.

If terms are changed during the analysis process (e.g. stemmed or stop words being removed), how do searches work? Find the answer in this lecture.

Let's go through the most important built-in analyzers and also cover how to configure and use them.

In this lecture, we are going to build a custom analyzer. As part of that, I will show you how to configure a built-in token filter as well.

Apart from specifying analyzers when creating a new index, sometimes we might need to add an analyzer to an existing index. Let's see how to do that in this lecture. In the process, we will look at opening and closing indices, as well as static and dynamic index settings.

Let's take a look at how to update analyzers. While doing so is easy, there is a very important thing to be aware of; failing to handle the situation correctly, can potentially lead to big problems.

A quick wrap up of this section.

A quick recommendation in regards to types.

Quick rundown of the two methods of searching data in Elasticsearch; request URI searches (query string), and the Query DSL.

A few examples of searching for data with query string searches (request URI).

An introduction to searching with the Query DSL.

Explanation of how a search query is handled within an Elasticsearch cluster.

Taking a closer look at search query results.

An explanation of how relevance scores are calculated.

See how to use the Explain API to debug why a given did or didn't match a given document.

Understand the two contexts in which search queries can be executed in Elasticsearch and what the differences are.

Before diving into the various categories of search queries, let's take a look at the difference between full-text queries and term level queries.

Test your understanding of the basics of searching for data in Elasticsearch.

Introduction to the concept of term level queries and when to use them.

Let's start out simple by searching for a term with the "term" query.

In this lecture, we search for multiple terms with the "terms" query.

This lecture shows how to retrieve multiple documents at once by IDs.

This lecture covers matching documents with range values.

This lecture shows how to use date math and relative dates when searching date fields.

See how to match documents that contain non-null values. In this lecture, we search for documents containing at least one tag.

See how to match documents where a given field's value begins with a specified prefix.

Searching with wildcards, being a question mark or an asterisk.

See how to supply a regular expression that is then used for matching field values.

Introduction to what full text queries are all about, and an introduction to the new test data that we will be working with throughout this section of the course.

Learn the fundamentals of the "match" query, which is probably the most essential query to understand in Elasticsearch.

This lecture covers how to search for phrases in Elasticsearch.

In this lecture, you will learn how to search multiple fields with the same query.

Introduction to what compound queries are about.

See how to write boolean queries with the "bool" query and understand how relevance scores are affected.

See how named queries can be used to debug and possibly understand the relevance of matching documents.

You already saw how to use the "match" query, but let's dive a little deeper and talk about how it works internally.

Introduction to this section on joining queries.

See how to query nested objects with the nested type. Also learn how this works internally and why this cannot be done without the "nested" query type.

See how to figure out which nested objects caused a document to match a given query.

Learn how to map document relationships with the "join" field data type.

Learn how to add documents for indices with "join relationships."

Learn how to rRetrieve documents based on their parent's ID.

See how to search for documents whose parent match certain criteria.

See how to search for documents whose child documents match certain criteria.

Learn how to add and use multi-level relationships, also referred to as nested relations.

See how to include inner hits with queries using a "join" field.

See how to use the so-called terms lookup mechanism. Essentially a more efficient way of using the "terms" query when dealing with lots of terms.

Joining queries are great in some scenarios, but not so great in others. This lecture discusses the pros and cons of using join queries and gives you recommendations about when to use them and when to stay away from them.

Joining queries are often not very fast. This lecture discusses the performance of joining queries and teaches you when performance is going to be an issue, enabling you to determine whether or not the "join" field is the right tool for your particular use case.

See how to choose between JSON and YAML as the result formats, and also how to format JSON for human eyes.

Learn how to specify which fields Elasticsearch should return for each match.

See how to specify how many search results Elasticsearch should return for a request.

See how to specify a match offset for a request.

See how the result size and offset can be combined to implement pagination within an application.

This lecture shows how to sort results in ascending and descending order.

This lecture shows how to sort by multi-value fields, such as average ratings, for example.

See how to apply filters to queries.