Apache Druid for Data Engineers (Hands-On)

Name: Apache Druid for Data Engineers (Hands-On)
Rating: 2.5 (9 reviews)

Learn everything about Apache Druid a modern real-time analytics database.

Role Play

Created byBigdata Engineer

Last updated 5/2026

English

English [Auto],

What you'll learn

Understand the fundamentals of real-time analytics databases and why Apache Druid is unique.
Explore key features, technology stack, and use cases of Druid.
Install Apache Druid on a Linux environment and on Windows using Docker Desktop.
Navigate the Druid web console to load, query, and manage data interactively.
Understand the architecture of Druid, including servers, services, and external dependencies.
Learn how Druid organizes data with datasources, segments, and identifiers.
Load data into Druid from local files, URIs, and real-time Kafka streams.
Run queries and explain plans, aggregate data with rollups, and optimize query performance.
Compare Druid with data warehouses (Redshift, BigQuery), search systems (Elasticsearch), and time-series databases.
Answer common FAQs around deployment, memory, compute, and integration with other tools.

Course content

9 sections • 47 lectures • 2h 43m total length

Introduction to Course8:57
Explore Apache Druid, an open-source, high-performance real-time analytics database, learn to load data, run subsecond queries, and understand its architecture with historical, real-time, and coordinate nodes.
Real-time Analytics Databases1:21
Analyze data in real time as it enters the database to store, process, and retrieve data, enabling rapid, data-driven decisions.
What is Apache Druid?2:15
Explore Apache Druid as a real-time analytics database built for fast slice-and-dice OLAP queries, with real-time ingestion, high uptime, and fast aggregation for event-oriented data.
Key Features of Druid5:18
Explore Apache Druid's key features: columnar storage, scalable architecture, real-time or batch ingestion, and fast filtering via bitmap indexes. Understand time-based partitioning and automatic summarization at ingest for efficiency.
Technology9:57
Use cases10:32
Leverage Apache Druid for real-time ingestion and fast query performance with high uptime across diverse use cases. Analyze clickstream, network telemetry, iot, and application metrics for interactive olap and bi.
When to use Druid2:01
Discover when to use Druid for high insert rates with infrequent updates, fast aggregation and reporting queries, time-based optimization, and data loading from Kafka, DFS, flat files, or S3.
When not to use Druid1:21
List of Company using Apache Druid3:44

Installation5:56
Install apache druid on your local machine by creating a directory, downloading the tarball with wget, extracting it, and noting that Windows is not supported and Java 8/11/17 are required.
Start up Druid services3:56
Start up Apache Druid services using the automatic single machine configuration with default settings; run bin/start-druid, then view the web console and stop with Ctrl-C.
Open the web console1:53
Explore starting Apache Druid via nano quickstart after microservices issues, view the web console at localhost:8888, and review core Druid services like ZooKeeper, middle manager, historical, router, broker, and coordinator.
Load data6:35
Query data3:04
Overview of the Druid Web Console Part 16:44
Overview of the Druid Web Console Part 23:37
Discover the Apache Druid web console’s data ingestion options, including streaming and batch sources, plus basic visualizations like time charts, tables, and pie charts.

Installing Docker on Windows1:15
Downloading Apache Druid Image on Docker6:13
Learn how to run Apache Druid on Docker Desktop for Windows by pulling Apache Druid 32.0.1, Zookeeper latest, and Postgres latest images after meeting prerequisites.
Running Apache Druid on Docker Desktop5:40
Learn to run Apache Druid on Docker Desktop using Docker Compose and start services. Access localhost to run select star from schemata and stop with Docker Compose down.

Druid Architecture2:09
Explore Apache Druid's distributed, cloud-friendly architecture with independent, fault-tolerant servers and services, including master, query, and data servers, and dependencies like meta storage, zookeeper, and deep storage.
Druid Servers2:54
Explore how Druid servers are organized into master, query, and data roles, with master coordinating ingestion and availability, and query broker and router, and data historical and middle manager tasks.
Druid Services7:10
Discover how Apache druid services ensure data availability through the coordinator's segment management and replication, and how the broker, router, overload, and middle manager coordinate ingestion and queries with zookeeper.
Druid Services in Simple terms4:20
External Dependencies4:00
Learn how Apache Druid uses external dependencies, including deep storage, metadata storage, and zookeeper, to manage data, metadata, and cluster coordination.

Datasources and Segments3:11
Learn how Druid stores data in data sources, similar to tables in RDBMS, partitioned by time into chunks and segments, with each segment a compact, indexed file that accelerates queries.
Segment Identifiers0:46
Explore segment identifiers in Apache Druid, where each segment carries a multi-part key—data source name, time interval, segment granularity, version number, and partition number.

Introduction to Segments2:24
Explore how Apache Druid partitions data into time-based segments and chunks, creates segments per interval, and manages segment file sizes between 300 and 700 MB by adjusting granularity or partitioning.
Segment File Structure1:49
Explore the segment file structure in Apache Druid, a columnar format storing timestamp, dimension, and metrics separately to reduce latency by scanning only needed columns.

Load Data from Local Files6:43
Base directory: quickstart/tutorial/
File filter: wikiticker-2015-09-12-sampled.json.gz
Load Data from URI4:14
https://druid.apache.org/data/wikipedia.json.gz.
Load Data from Kafka (Prerequisite Introduction to Kafka)1:17
Discover how Apache Kafka uses a pub-sub architecture with producers, brokers, and consumers. Learn how messages flow through a central broker rather than direct producer-consumer connections.
Installing Single Node Kafka Cluster1:51
Change the following to avoid Zookeeper Issue conflict0:13
Load Data from Kafka Part 18:52
Download and unzip the data with curl and gunzip, start Zookeeper and Kafka, create a Kafka topic, and publish data so Apache Druid can fetch records from Kafka.
Load Data from Kafka Part 23:48
Learn to load data from Kafka into Druid by creating a streaming spec, configuring json input and timestamp, setting day granularity, publishing, and querying the KTM Kafka data source.
Query Data Explain Plan1:52
Write and run a Druid query on the Wikipedia table to count edits by page and country name, then view the explain plan using the explain query tool.
Aggregate data with rollup10:49
Learn how to aggregate data with roll up in Apache Druid by enabling roll up, setting minute-level granularity, and computing sums of packets and bytes across source and destination IPs.

Is Druid a data warehouse? When should I use Druid over Redshift/BigQuery0:50
Is Druid log aggregation/log search system? When should I use Druid over Elastic0:32
Is Druid a timeseries database? When should I use Druid0:31
Does Druid separate storage and compute?0:12
How is Druid deployed?0:17
Where does Druid fit in my big data stack?0:27
Is Druid in-memory?0:41
Do clients ever contact the Coordinator process?0:18
Does it matter if Coordinator process starts up before or after other processes?0:23

Requirements

Basic understanding of databases – familiarity with concepts like tables, queries, and indexing will help.
Knowledge of SQL – since querying data in Druid often uses SQL-like syntax.
Basic Linux command-line skills – useful for installation and managing services.
Familiarity with Docker (optional) – helpful for the Windows setup section, but step-by-step guidance is provided.
Some exposure to Big Data or Analytics tools (optional) – knowledge of tools like Kafka, Spark, or data warehouses will give extra context, but it’s not mandatory.
Eagerness to learn real-time analytics – no prior experience with Apache Druid is required; the course starts from the basics.

Description

Apache Druid for Data Engineers (Hands-On)

Modern data-driven organizations need real-time insights from massive streams of events, logs, and metrics. Traditional data warehouses and batch processing systems struggle to deliver sub-second query performance on high-volume, high-velocity data. This is where Apache Druid comes in.

Apache Druid is a high-performance real-time analytics database widely adopted by companies like Netflix, Airbnb, Lyft, and Cisco for powering interactive dashboards, anomaly detection, log analytics, and user-facing applications. Designed for speed and scalability, Druid combines the best of OLAP databases, time-series stores, and search systems.

This course, Apache Druid for Data Engineers (Hands-On), is a step-by-step, practical guide that takes you from installation to real-world use cases. You’ll learn how to install Druid on both Linux and Windows (via Docker), explore its architecture, storage design, and segment structure, and practice loading and querying data from local files, URIs, and Kafka streams. You’ll also gain clarity on where Druid fits in the modern big data stack by comparing it with Redshift, BigQuery, and Elasticsearch.

What You’ll Learn

By the end of this course, you will be able to:

Understand the fundamentals of real-time analytics databases and why Apache Druid is unique.
Explore key features, technology stack, and use cases of Druid.
Install Apache Druid on a Linux environment and on Windows using Docker Desktop.
Navigate the Druid web console to load, query, and manage data interactively.
Understand the architecture of Druid, including servers, services, and external dependencies.
Learn how Druid organizes data with datasources, segments, and identifiers.
Load data into Druid from local files, URIs, and real-time Kafka streams.
Run queries and explain plans, aggregate data with rollups, and optimize query performance.
Compare Druid with data warehouses (Redshift, BigQuery), search systems (Elasticsearch), and time-series databases.
Answer common FAQs around deployment, memory, compute, and integration with other tools.

Who this course is for:

Data Engineers & Big Data Developers who want to master real-time analytics databases.
Data Analysts & BI Professionals looking to build sub-second interactive dashboards on streaming data.
Software Engineers integrating analytics into user-facing applications.
Students & Enthusiasts who want to learn how modern analytics systems like Druid power platforms at Netflix, Airbnb, and Lyft.

Apache Druid for Data Engineers (Hands-On)

What you'll learn

Explore related topics

Course content

Introduction9 lectures • 45min

Installation of Apache Druid on Linux Machine7 lectures • 32min

Installing Apache Druid on Windows Machine using Docker Desktop3 lectures • 13min

Architecture of Druid5 lectures • 21min

Storage Design2 lectures • 4min

Segments2 lectures • 4min

Data Loading in Druid9 lectures • 40min

Frequently Asked Questions9 lectures • 4min

Practice Test2 lectures • 1min

Requirements

Description

Who this course is for: