
Discover how Kafka producers publish data to topics and partitions via partition leaders, use round-robin when keys are absent, and manage acknowledgments to balance durability and latency.
Learn how Kafka consumers read data from topics across partitions, coordinate in groups, manage offsets, and choose delivery semantics such as at most once, at least once, and exactly once.
Understand distributed systems with brokers and clusters, where producers and consumers exchange data through replication, partitioning, and consensus, coordinated by ZooKeeper and gossip protocols in Kafka-like setups.
Learn to start a basic Kafka cluster using docker-compose by defining zookeeper and broker services, configuring environment variables and advertised listeners, and managing containers with docker compose down.
Explore console consumer in Kafka: set up brokers and ZooKeeper, create topics, run producer and multiple consumers, observe consumer groups, offsets, and lag to understand data flow.
Understand partition concurrency by starting multiple consumer instances, each reading from a distinct partition to preserve order. Align the number of consumers with the partition count for true parallelism.
More than 80% of all Fortune 100 companies trust, and use Kafka.
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
Application of Kafka
Apache Kafka is based on the commit log, and it allows users to subscribe to it and publish data to any number of systems or real-time applications. Example applications include managing passenger and driver matching at Uber, providing real-time analytics and predictive maintenance for British Gas smart home, and performing numerous real-time services across all of LinkedIn.
About the Course
I am creating Apache Kafka for absolute beginners course to help you understand the Apache Kafka Internal, the architecture of Kafka distributed system, Role of Zookeeper, Kafka CLI and Client With NodeJS.
We'll learn all the fundamentals of Apache Kafka in simple way with NodeJS.
At the end of this course, you'll be in good state to work on apache kafka project:
The Apache Kafka Architecture
Kafka Fundamental topic: Topics, Partitions, Brokers, Replicas, Producers, Consumers and Zookeeper
Launch your own Kafka cluster with docker – Windows / MacOS X / Linux
Kafka Command Line Interface (CLI)
Code Producer and Consumers using the NodeJS (KafkaJS Library)
Learn How kafka Work as distributed system
What is Message Broker
Kafka Client Configuration
Producer Configuration in detail
Transaction
Batch Processing
Learn How to setup Logger With kafka Client
etc...