
Explore how data layout, indexing, tokenizers, and analyzers shape search performance using a wardrobe analogy, showing how color, type, or size layouts affect retrieval time, redundancy, and scalability in OpenSearch.
Extend the minimal text search workflow in OpenSearch and Elasticsearch. Demonstrate indexing, using keyword fields, running aggregations, and checking cluster health via dashboards and APIs.
Explore edge n-grams in OpenSearch by building an autocomplete analyzer, configuring min and max grams, indexing with product name mappings, and testing trigram and unigram matches.
Walk through registering and deploying a local torchscript sentence transformer model for OpenSearch text embedding, highlighting a torch vs onnx format deployment error and related troubleshooting.
Explore sparse encoding in OpenSearch by registering, deploying, and using sparse encoding models to generate token: weight pairs in sparse vectors.
Explore how agentic frameworks in OpenSearch coordinate specialized tools and large language models through flow, conversational flow, and conversational agents, using retrieval augmented generation (rag) with vector embeddings.
Register and deploy a cross-encoder model and use a rerank pipeline to order results by semantic similarity. Show that reranked results outperform search for queries like capital of United States.
Learn end-to-end data exploration in OpenSearch dashboards by adding sample data, using discover and dashboards, applying filters and boolean queries, and creating visualizations with dxl and dark mode customization.
Explore geospatial analysis with OpenSearch by indexing geo points and performing geo bounding box, distance, polygon, and shape queries using GeoJSON, with attention to precision and rounding.
Elasticsearch is a well-known search platform adopted in enterprises, SMBs and startups. Elasticsearch excels at lexical search use cases using BM25 algorithm , that is built on top of Lucene. However, with the advent of AI and large language models, Semantic Search, Hybrid Search, Neural Search, Multi-modal search etc. have become more of a norm than rarity.
OpenSearch (originally a fork of Elasticsearch started in 2021) has gained immense popularity and adoption in open source, and enterprise communities with its Apache open source license and a Linux foundation project. While providing parity with all the lexical search capabilities of elasticsearch, OpenSearch integrates with LLM models (e.g. sentence transformers) , providers like OpenAI, Cohere, Anthropic and defines agentic workflows. As a win, Oracle switched to OpenSearch for its PeopleSoft search capabilities. AWS provides Opensearch-as-a-service on its cloud and that already speaks to the production readiness.
AI & ML Search with OpenSearch course provides end-end training on installing, configuring and understanding OpenSearch , while implementing real search use cases like retrieval-augmented-generation (RAG), agentic workflows and migrating from Elasticsearch to OpenSearch. Emphasis has been laid on AI/ML use cases more than the traditional/lexical concepts, though the latter is covered for historical context.
To compare Elasticsearch (ELK stack) & OpenSearch, we can roughly equate the below:
Elasticsearch ~ OpenSearch
Logstash ~ Data Prepper
Kibana ~ OpenSearch Dashboards
OpenSearch is a fast moving platform in terms of its releases and features. We will be using version 2.17 which is production-ready as of September 2024. Docker has been extensively used in the course to ensure execution reproducibility of the entire course code.
I am excited to be your instructor and hoping you resonate the same excitement !