AI & ML Search With OpenSearch (Intermediate level)

Set up an OpenSearch single-node server with docker-compose, configure security options, access via Kibana on port 5601, load sample data, and verify indices.

Install and configure OpenSearch with Docker Compose, from a single node with no security to a cluster. Learn login, Kibana access, cluster mode, and GPU and CPU options for embeddings.

Navigate traditional search in OpenSearch by following the documentation index, gaining a high-level map of text analysis, tokenization, ingestion, and the DSL, with paths to autocomplete and search-as-you-type.

Master traditional text search concepts in OpenSearch, including analyzers, tokenizers, text matching, and boolean queries. Apply these patterns to e-commerce, support tickets, CMS, and analytics with autocomplete and highlighting.

Explore text search concepts in OpenSearch through a hands-on ecommerce dataset demo, covering mapping, match all, pagination, scroll, search after, point-in-time, sorting, and highlight.

Explore autocomplete concepts in OpenSearch, defining edge n-grams at ingest time and using a standard analyzer at search time for accurate prefix matching, plus completion and term suggesters.

Learn how to use OpenSearch phrase suggester with trigram shingles to handle typos and phrase matching, including retrieving specific stored fields, match phrase with slop, and exact matches via keyword.

Explore proximity search and advanced OpenSearch concepts, including match phrase with slop, term queries, filters, bool queries with must, should, and must not, plus pagination, sorting, and named queries.

Navigate section 3 of AI and ML search with OpenSearch, exploring ingest pipelines, data processing, and vector basics to enable production-level indexing and querying.

Explore how OpenSearch ingest pipelines apply a sequence of processors to transform documents before indexing, compare them to Data Prepper, and handle failures with OnFailure and IgnoreFailure options.

Explore OpenSearch ingest pipelines by walking through about 43 ingest processors, including append, bytes, copy, csv, date, grok, and json processors, with practical code examples.

Explore ingest processors in OpenSearch, including split and join, case changes, renaming, removing, and gsub, plus html strip, url decode, sort, fingerprint, key value, and dissect.

Design ingest processors in OpenSearch, using for-each and inner and outer pipeline patterns. Enrich logs with GeoIP and IP2Geo, following best practices for modular pipelines and error handling.

Discover how to deploy and use a text embedding processor in ingest pipelines with OpenSearch to generate embeddings for job title and content, enabling semantic search.

Explore how OpenSearch search pipelines and processors transform queries and results through request and response processors, including filter query, ML inference, and re-ranking with practical demos.

Explore end-to-end ingestion and search in OpenSearch, comparing no vectors, manual vectors, and automatic embeddings via ingest pipelines; tune with semantic, hybrid search, and BM25, evaluating precision versus recall.

Explore bulk ingestion and search concepts by applying KNN semantic search, hybrid search, and relevance tuning to rank results using vector embeddings, neural query, and field boosts.

Explore vector basics and semantic search with OpenSearch, contrasting sparse and dense embeddings, and see how sentence transformers, BERT, and cosine similarity enable semantic matching and clustering.

Explores vector basics, engines, spaces, and search techniques in OpenSearch, covering K-NN, semantic, hybrid, multi-modal searches, RAG, and vector quantization for scalable retrieval.

Explore kNN methods for vector search in OpenSearch, including hnsw and ivf, and compare engines such as FICE and Lucene to balance latency and memory.

Learn to generate vectors or embeddings from data in OpenSearch. See authentication, deploy a sentence transformer model with ML Commons, create an ingest pipeline, and enable a KNN index.

Explore data prepper as a scalable OpenSearch-native ingestion engine processing Apache and OTEL logs with grok, feeding structured data into OpenSearch dashboards.

Accelerate ingestion with OpenSearch by tuning the bulk API, batch size, and async processing to boost throughput and reduce disk usage, while applying translog and heap optimizations.

Apply end-to-end ingestion optimizations for OpenSearch, including jvm heap tuning, translog and segment replication, and zstd compression to boost bulk ingestion speed and cut latency.

Ingest data into OpenSearch by vectorizing fields, creating mappings, and using bulk async embeddings, then explore snapshots and production optimizations with docker-compose-fully-optimized.yml.

Navigate semantic, hybrid, and neural sparse search in OpenSearch, mastering meaning-based retrieval, Q&A, document similarity, and cross-lingual use cases with practical vector and text indexing, scoring, and re-ranking techniques.

Explore semantic search architecture using embeddings and vector indices, perform KNN and hybrid searches, and compare semantic versus keyword retrieval with OpenSearch.

Explore hybrid search in OpenSearch by blending keyword and semantic queries, using min-max normalization and 30/70 weighting of keyword and vector embeddings to rank results, with sorting and aggregations.

Discover neural sparse search in OpenSearch, blending sparse token weights with neural embeddings to speed results and reduce memory, using rank features, prune ratios, and ANN mode.

Explore specialized vector searches with OpenSearch, including nested kNN search with inner hits, radial search for distance or similarity thresholds, and MMR to balance relevance and diversity.

Explore llm agents in rag with OpenSearch, covering embeddings, semantic search, and deploying models on premises or via providers to power rag flows and multi-step agents.

Explore text embedding with OpenSearch by registering pre-registered sentence transformer models and deploying custom embeddings, comparing TorchScript and ONNX, and performing inference on embedded text.

Register, deploy, and test OpenSearch text embedding models using TorchScript and ONNX from local and Hugging Face hub, generating embeddings with sentence transformers.

Demonstrates sparse encoding and cross-encoder workflows in OpenSearch, compares sparse and dense embeddings, and shows end-to-end deployment, encoding, indexing, and semantic highlighting for improved neural search results.

Explore OpenSearch custom models and deploy a QA model like distilbert to answer questions from a retrieved context. Learn about the OpenSearch client setup, model registration, and embedding-driven retrieval workflows.

Learn how OpenSearch connects externally hosted models from Anthropic, DeepSeek, OLAMA, and OpenAI via connectors and ML Commons, enabling chat, embeddings, and batch ingestion workflows.

Explore guardrails for external OpenAI models in OpenSearch, including guardrail model groups and connectors. See how stop words guardrails and multi-modal image-to-text with GPT-4-O-Mini work, plus OLAMA mentions.

Explore how Olama enables locally deployed large language models with OpenSearch, preserving data privacy, by using dockerized models, embedding and chat capabilities, and a practical connector workflow.

Explore agents and tools with Ollama and OpenAI to power OpenSearch-based Q&A using vectorDB and embedding models. Demonstrate docker setup, indexing, and a rag flow with memory.

Explore the plan, execute, reflect (PER) agent pattern in OpenSearch, using vector db and ML model tools to plan steps, execute tasks, reflect on results, and refine answers.

Explore the theory and demo of agent tools in OpenSearch, including folders, notebooks, and helper functions, with OpenAI as the external model and color-coded workflow stages (input, processing, results).

Learn how the agent tool enables orchestrator–base agent calls to build multi-agent workflows in OpenSearch and how the web search tool with DuckDuckGo augments real-time results via ml model tool.

Demonstrates MLModel for remote llm inference with OpenSearch, dense VectorDB, neural sparse search, and tools like list and map, plus index mapping and health checks.

Explore log pattern tools, baseline versus selection analysis, anomaly detection with trace IDs, PPL queries, visualization, and scratchpad memory for agent context.

Explore retrieval augmented generation with a memory-driven conversational flow agent using open search. Learn how embeddings, long-term memory, and multi-knowledge bases power context-rich llm responses.

Explore the RAC conversational flow with multiple knowledge bases, dynamic index, memory, and hybrid RAG patterns using BM25, neural search, and RRF to generate unified responses.

Explore the model context protocol (MCP) with OpenSearch's inbuilt server and Langchain agent, and examine tool discovery, streaming transport, and read operations on indices and documents.

Navigate nine real-time OpenSearch projects, from search-as-you-type UIs to BI agents, with end-to-end FastAPI back-ends, WebSocket updates, and front-ends such as Streamlit, Gradio, and React.

Build a real-time search as you type app using Streamlit, Gradio, or React with a FastAPI backend and OpenSearch, featuring prefix matching, fuzzy search, typos, and highlighted results.

Benchmark ingestion and search in OpenSearch by comparing analyzers, BM25, and neural search on the SQuAD dataset, measuring ingestion throughput, latency, and relevance to guide hybrid strategies.

Explore geospatial data in OpenSearch by working with geopoints and geoshapes, using GeoJSON coordinates, geobounding box and geodistance queries, and polygon-based shapes to locate cities and boundaries.

Launch the Gradio learning app to explore geospatial queries, geopoints, geoshapes, and GeoJSON formats with an OpenSearch cluster and a UI-driven learning flow.

Learn to build a non-RAG business intelligence agent that translates natural language queries into SQL for MS SQL or Postgres, grounding context with metadata and generating insights within guardrails.

Build a business intelligence agent with retrieval-augmented generation using OpenSearch for MSSQL metadata. Ingest column and table descriptions and enable semantic hybrid search to generate SQL, visualize, and derive insights.

Build a business intelligence agent with RAG on Postgres using AdventureWorks, OpenSearch vectors, and metadata-driven context to generate SQL from natural language queries.

Build an end-to-end business intelligence app with OpenSearch and Postgres, extracting metadata, generating AI descriptions, ingesting vectors, executing SQL, visualizing results, and deriving insights.

Develop a full-stack business intelligence app with conversational memory by ingesting metadata into OpenSearch and handling first-time versus follow-up queries using last-five interactions.

Explore end-to-end full-stack BI agent apps with long-term memory, a vector field and ingest pipeline, and neural search to cache queries and reduce round trips to the LLM.