Elasticsearch 9 & Spring Boot: Search Engine Masterclass

Set up Elasticsearch and Kibana with docker compose, run a single-node Elasticsearch exposing 9200 and 9300, access Kibana on 5601, and perform CRUD via REST APIs for learning.

Explore the Kibana dev tools console to query Elasticsearch using REST APIs, including _cluster health and _nodes, view status (green), and use curl and CAT API endpoints for compact outputs.

Explore the core concepts of Elasticsearch, including indices as storage units, documents as records, and fields as columns, while performing CRUD to understand indexing and retrieval.

Use the Elasticsearch index API in Kibana dev tools to list indices, filter by pattern, create a products index with PUT, and delete it, noting yellow cluster health.

Learn how to perform crud by adding documents to an Elasticsearch index with a post to the /_doc endpoint using JSON, with optional mapping and auto or custom IDs.

Explore how Elasticsearch stores documents with the underscore source field and metadata fields, and how index, ID, version, sequence number, and primary term enable optimistic locking.

Learn to store documents with your own IDs in Elasticsearch using the Kibana dev console, including creating the books index and inserting documents with IDs like 1, 2, 3, 4.

Learn to read documents in Elasticsearch by id or fetch all documents from an index using Kibana dev console. Understand how _source and hits appear in get and search responses.

Retrieve a document with a get request, then update using PUT or POST with upsert behavior, replacing the whole object or patching a single field like year.

Update specific fields in Elasticsearch via the update endpoint using post, specifying the id and doc field to replace values, and explore upsert behavior for missing ids.

Learn scripted updates in Elasticsearch to adjust document fields, like increasing price by one, using the script field and ctx, and weigh pros and cons.

Learn to delete a document in Elasticsearch by using the delete request with a book ID, verify not found, and remove the index if needed.

Discover how Elasticsearch uses Lucene to build an inverted index by tokenizing documents into terms and mapping them to document IDs and positions.

Explore how the term dictionary fits into the inverted index, and how term frequency, document frequency, and posting lists shape document relevance in Elasticsearch.

Explore how Elasticsearch uses immutable segments to store data, with segment-level inverted indexes, memory buffers, and periodic merges that optimize write speed while balancing slower search and marker-based deletions.

Elasticsearch defers newly indexed documents by default, making them searchable only after a short refresh; use the refresh API to see them, especially during bulk inserts in Spring Boot.

Explore how Elasticsearch uses Apache Lucene to index and tokenize documents, creating an inverted index, and learn crud operations for an index and its documents, including upserts and scripted updates.

Learn how Elasticsearch clusters use sharding and replication to achieve high availability and scalability, deploying multi-node setups with Docker Compose and understanding cluster coordination on port 9300.

Learn how to scale Elasticsearch by sharding a large products index into multiple primary shards, distributing data across nodes, and understanding the distinction between shards and Lucene segments.

Discover how Elasticsearch routes requests in a cluster, distributes data across shards using a routing key and id, and uses scatter-gather for id-less searches.

Explain how primary shards distribute data and replica shards provide high availability by backing up data on separate nodes, and how replicas promote to primary and serve search requests.

Learn index creation with shard and replica settings, diagnose yellow cluster health on a single node, and configure primary shards with or without replicas to explore performance.

Explore Elasticsearch node roles, including master election, data, coordinating, and ingest; learn how role assignment supports high availability, indexing, search, data replication, and hot, warm, cold tiers.

Set up a three-node Elasticsearch cluster with Docker Compose (es01, es02, es03), configure initial master and discovery, allocate memory and CPU, and monitor Kibana to ensure the cluster is green.

Explore how master election works in a three-node Elasticsearch cluster, perform docker compose stop/start, and understand voting needed for cluster leadership.

Demonstrates high availability through shard replication, with one primary and one replica, coordinated across es01, es02, and es03, and how failover promotes replicas and reassigns shards.

Demonstrates creating an index with two primary shards and one replica in a three-node cluster, inserting four documents, and performing a distributed search that returns a combined result.

Configure a three-node elasticsearch cluster with es01 as initial master and discovery seed host property; when es01 goes down, es02 or es03 become master and get nodes api shows roles.

Demonstrates configuring a three-node Elasticsearch cluster with dedicated master and data nodes, showing master election, node roles, and voting, plus resilience when the master goes down.

Set a node's roles to an empty list to make it a coordination only node, preventing it from acting as master or data in the Elasticsearch cluster.

Master optimistic concurrency control in Elasticsearch by using sequence numbers and primary terms to safely update documents and prevent race conditions.

Explore cluster settings in Elasticsearch masterclass, learn how persistent and transient settings control index auto_create_index, and use the cluster settings API to customize defaults.

Determine the right number of primary shards based on application and data needs. Plan 500 GB with 10-20 shards; each shard handles 25-50 GB and avoids burdening the cluster master.

Learn why you cannot change the primary shard count of an existing Elasticsearch index, and how to migrate to a new index with the desired shards through reindexing.

determine the recommended replica shards count for production to ensure high availability, and consider maintenance scenarios where multiple nodes fail, prompting replicas equal to the expected failures.

Understand how clustering, sharding, and replication keep Elasticsearch highly available and scalable, with primary and replica shards, indexing, and failover ensuring safe, high-throughput updates.

Explore the elasticsearch bulk api for performing multiple create, update, and delete operations in a single post to the /bulk endpoint using ndjson payloads with line-by-line parsing.

Demonstrate the Elasticsearch bulk API in Kibana Dev Console by performing bulk inserts with and without IDs, and by executing create, update, and delete operations within a single NDJSON payload.

Explore how the Elasticsearch bulk API handles errors gracefully within a single request. See mixed outcomes, where some items succeed, a 404 document missing exception, and others complete.

demonstrates optimistic concurrency control in bulk updates using sequence number and primary term to prevent race conditions across indices, with multi-index bulk actions and cross-index searches.

Demonstrates bulk API file upload for importing ndjson data into an Elasticsearch instance using a curl post request, with a newline and port guidance; verify results in Kibana dev console.

Copy documents from an old index to a new index with the reindex api, preserving the original data and applying shard settings. Customize with selected fields and multiple source indices.

Use the bulk API to add or update hundreds of documents in a single NDJSON request. Reindex data between indices with selective fields and a query to filter the documents.

Explore how analyzers process unstructured data to enable fast search in Elasticsearch. Compare this with relational databases and SQL, and learn how data mapping and analyzer components support querying.

Explore how Elasticsearch analyzers process text during indexing and searching by using character filters to strip HTML, tokenizers to split, and token filters to lowercase and refine tokens.

Clarify how analyzers process each field like name, description, and features, generating tokens, updating the inverted index, and lowercasing colors to support efficient search.

Explore how the Elasticsearch analyze API simulates text analysis for indexing and searching, using character filters like HTML strip, mapping, and pattern replace to transform text and reveal tokens.

Master Elasticsearch tokenizers, including the standard tokenizer and uax url email, learn how HTML strip and character filters shape token output in Kibana.

Explore token filters in Elasticsearch, configuring lowercase and uppercase, length-based and unique token filters with character filters, tokenizers, and a Kibana workflow.

Explore how the synonym filter normalizes terms like couch and sofa to the same token in an Elasticsearch pipeline, enabling the search engine to return relevant documents.

Enable the stopword filter to remove common articles and frequent words from indexed text, improving search relevance and reducing index size, as shown with Kibana and the standard tokenizer.

Explore stemming in Elasticsearch to reduce words to base forms with the Porter stemmer, improving the inverted index with examples like cooking to cook and ponies to pony.

Explore how Elasticsearch analyzers configure tokenization and filters, comparing standard, simple, whitespace, stop, and keyword analyzers. See the analyze API and Kibana demos to understand tokenizers and filters.

Learn to build a custom analyzer inside an index by configuring HTML strip character filter, standard tokenizer, and uppercase filter, then analyze text using the index analyze API.

Elasticsearch performs analysis, tokenization, and inverted indexing for search, so inserts can seem slow versus RDBMS; but sharding scales indexing, while writes go to the database and searches rely on Elasticsearch.

Elasticsearch serves as a search engine, not the source of truth, with data flowing from the product service's Postgres via Kafka to Elasticsearch in bulk.

Discusses why custom java-based analyzers are rare, and how normalization via a separate app supports Elasticsearch indexing with full control over character and token filters.

Analyze how analyzers split text into tokens for indexing and searching using character filters, tokenizers, and token filters. See how HTML strip and pattern replace affect tokens.

Define fields and data types in Elasticsearch via dynamic or explicit mapping. Compare text versus keyword fields, tokenization, and BKD trees for range queries on numeric and date data.

Explore dynamic mapping in Elasticsearch, see how indices and mappings are inferred in Kibana, and learn potential pitfalls that prompt using explicit mappings in production.

Explore explicit mapping to create an index with settings and mappings, define fields like name (text), age (integer), and email (keyword), and understand how tokenization affects searches.

Continue the demo by inserting a document with a new field like phone number; Elasticsearch infers long and you can add city as text to the index mapping.

Elasticsearch does not allow changing an existing field type, such as switching email from keyword to text; use the reindex API or delete and recreate the index.

Elasticsearch makes every field optional in the mapping, so documents may include, omit, or set fields to null, and type constraints like integer can be single values or lists.

Learn to configure a custom analyzer in Elasticsearch, apply it to the blog body field, and improve search results with lowercase and stemmer filters, using Kibana for testing.

Configure your custom analyzer at the field level to reuse it for the title field, since no default analyzer exists; use a map of multiple custom analyzers for each field.

Skip indexing unused fields in Elasticsearch to save CPU, memory, and disk space, speeding up indexing. Use application-side filtering or mapping with enabled false.

Master 1-to-1 inner object mapping in Elasticsearch by modeling a movie with a director, using dot notation for properties, and testing with bulk inserts and Kibana searches.

Apply nested object mapping to preserve one-to-many relationships in Elasticsearch, using a nested actors field with name and role, testing with bulk inserts and searches.

Define data mapping in Elasticsearch with dynamic and explicit approaches; prefer explicit mapping for production. Explore common field types (text, keyword, numeric, date) and nested mappings for inner objects.

Learn how Elasticsearch handles full text search and relevance with the query DSL in JSON, including simple and complex queries, endpoints, index setup, and test data.

Explore the match all query, similar to select all SQL, by creating a products index, defining a mapping, inserting seven records, and executing the query via Kibana.

Learn how to select specific documents in elasticsearch by providing an IDs list in the request body, returning only the records matching chosen IDs (for example 1 and 4).

Understand the term query for exact matches, why it targets keyword fields over text fields, and how to enable case-insensitive matching for multiple values like blue or black.

Explore range queries by filtering price with less than, less than or equal to, greater than or equal to, and between values, using term query for exact matches.

Master prefix, wildcard, and regexp queries in Elasticsearch, learning when to use starts with, ends with, and complex patterns, and understanding case sensitivity and performance trade-offs.

Learn how the exists query checks non-null fields like brand, using a mapping, to filter records; this demo shows creating an index, inserting data, and counting five matches.

Explore how relevance ranks Elasticsearch search results by matching user intent, using term frequency and document frequency to calculate relevance scores and tweak them for business needs.

Explore how the match query scores relevant documents in Elasticsearch using an articles index and content field. See how tokenization, term frequency, document length, and field length influence relevance.

Explore how the Elasticsearch match query computes relevance scores, compares spring and boot, and uses and/or logic in Kibana with live document indexing and scoring insights.

Learn how Elasticsearch fuzziness handles misspellings and variations using edit-distance levels. See how one or two edits affect results, inverted indexes, and the prefix length to balance precision and relevance.

Explore how the match phrase query requires spring and season to appear together in the content field, contrast with the and operator, and tune with slop to allow word gaps.

Explore how multi match searches across multiple fields, including title and body, with weighting, tiebreaker, and options like fuzziness and phrase with slop to improve relevance.

Explore how Elasticsearch highlights matching terms using the highlight feature, with pre and post tags, field targeting (body and title), Kibana demos, and match-phrase queries with slop.

Discover multi-field mapping, storing a field as both text and keyword to support fuzzy and phrase searches as well as exact matches and aggregations.

Explore leaf and compound queries in Elasticsearch, contrasting strict and flexible searches, with leaf queries like term on brand and range on price, then build complex conditions.

Learn to build complex queries with the Elasticsearch bool query using filter, must, must not, and should clauses, including range queries for price and rating and scoring relevance.

Explore building a bool query to filter a products index by price under 100 and rating, and exclude Nike with must not, noting score behavior.

Explore the should clause in Elasticsearch bool queries, comparing its scoring with must and filter. Learn when should filters documents and how it contributes to score in multi-condition queries.

Explain how the bool query's should clause creates an or-like filter with boosting, how must and filter interact, and how to prioritize tennis or running shoes by score.

Outline a four-step approach to Elasticsearch bool queries: group conditions in filter, move nots to must_not, move should to should, and place text queries in must for relevance.

Learn how to craft a bool query for a products index using match, range, must, and should clauses to filter shoes by name, price, and rating, including under $50.

Learn to construct a composite Elasticsearch bool query by combining two inner queries: shoes priced under 60 and boots with a rating of at least 4.5, using should clauses.

Explain how minimum should match works in an Elasticsearch bool query, using a Nike product filter with should conditions and rating or price constraints.

Explore how the should clause in Elasticsearch boosts personalized recommendations by scoring conditions like highly rated or action movies, using user data to rank results on large indexes.

Demote documents with a boosting query by applying a negative boost to their scores. The approach uses a match query and conditions like winter boots, and can model Netflix recommendations.

Leverage the disjunction max query to combine multiple match queries, boost key terms with tiebreakers, and handle synonyms for broader, more relevant spring-focused Elasticsearch searches.

Explore query string techniques to quickly debug and build complex Elasticsearch queries, including or and operators, phrase search, field-specific search, fuzziness, wildcard, and range filters.

Explore geospatial queries by creating a restaurants index, using the geo point type and geo distance query to find nearby restaurants based on user location.

Explore 1-to-1 and nested child objects in Elasticsearch, mapping and indexing, using term and bool queries, and avoid dot notation that breaks parent-child relationships.

Explore elasticsearch nested queries in a 1-to-many model where movies are the parent and actors are nested child documents, using path actors with match or term queries.

Explore the inner hits feature in Elasticsearch to identify which nested inner document satisfied a query, helping you debug when parent objects are returned due to many inner records.

Learn date math in Elasticsearch by building an orders index, using boolean fields, and using now-based queries and range filters (such as last month) to fetch data.

Master querying Elasticsearch with strict, flexible, and compound queries, including bool queries with filter, must, must not, and should, and learn how relevance scoring and nested mappings shape results.

Learn how to perform field selection in Elasticsearch using _source, including selecting specific fields, excluding fields, and returning only movie IDs or titles from a sample movies index in Kibana.

demonstrate elasticsearch pagination using size and offset to slice results, and apply a page formula from the first page starting at zero to show two-record pages.

Sort Elasticsearch results by relevance or by fields like price, date, and rating, with IMDb rating and year as examples. Use keyword or date fields and multi-field mapping for sorting.

Learn how to sort Elasticsearch documents by a nested actor birth date using sort mode max, selecting one date from each actor list and declaring the nested path.

Discover how to sort by id in Elasticsearch for Java Spring developers, why auto generated ids can't be sorted, and embed id in the body with number or keyword types.

Learn how aggregation works in Elasticsearch, applying grouped aggregations and functions like min, max, sum, average, and value count via the search API and query DSL.

Explore metric aggregation in Elasticsearch by computing max, min, count, and other stats on a price field, using aggregations, stats, and percentiles within a products index.

Explore bucket terms aggregation in Elasticsearch, using keyword fields and multi field mapping to implement group by facets for size, color, material and brand, and filters affect counts in Kibana.

Learn how bucket terms aggregation groups data by brand and applies per-group metrics such as average and sum of price using nested aggregations.

Learn bucket range and histogram aggregations in Elasticsearch by defining price ranges and an interval, and use min doc count to remove empty buckets.

Explore nested aggregations for stores with nested products, computing price stats (min, max, avg, sum) and grouping by product category via the nested path products.

Master Elasticsearch aggregation by using metric and bucket terms aggregations, group by keyword fields to build facets, and apply aggregates within groups, with range and histogram options for numeric distributions.