Apache Kafka Series - Kafka Monitoring & Operations
4.7 (480 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
6,716 students enrolled

Apache Kafka Series - Kafka Monitoring & Operations

Kafka Monitoring Setup with Prometheus and Grafana, Kafka Operations and Kafka Cluster Upgrades Hands-On. Setup in AWS
4.7 (480 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
6,715 students enrolled
Current price: $32.99 Original price: $129.99 Discount: 75% off
4 days left at this price!
30-Day Money-Back Guarantee
This course includes
  • 5 hours on-demand video
  • 13 articles
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Setup a Multi Broker Kafka Cluster in no-time in AWS (using CloudFormation)
  • Setup Administration Tools such as Kafka Manager, ZooNavigator, LinkedIn's Kafka Monitor
  • Setup Monitoring using Grafana and Prometheus
  • Learn how to perform a safe and automated Roll Restart of Kafka Brokers
  • Update Brokers Configurations in a safe way
  • Rebalance Partitions in a Kafka Cluster
  • Increase and Decrease the Replication Factor of Topics
  • Add a Broker to a Kafka Cluster
  • Service and Replace a Broker in a Kafka Cluster
  • Remove a Broker in a Kafka Cluster
  • Install Command Line Interface (CLI) tools to automate workflows
  • Upgrade a Kafka Cluster with no downtime
Requirements
  • Kafka Basics + Setup Knowledge
  • Linux Knowledge
  • Windows / Linux / MacOS X are ok
  • Cloud Computing Understanding preferred
  • No Knowledge of AWS CloudFormation is required
Description

One of the best courses in the Apache Kafka Series!

Learn Kafka Administration, Kafka Monitoring, Kafka Operations and Kafka Upgrades in this 5+ hours course. Join hundreds of knowledge savvy students into learning some of the most important knowledges that any Kafka administrator should know and master.

----------------------------

This course is important for the following reasons:

Administration for Apache Kafka

> Setup UI tools such as Kafka Manager, ZooNavigator and Kafka Monitor to get a full view of your cluster
> Understand basic operations you can perform with these tools

Monitoring for Apache Kafka

> A Kafka that is not monitored is a ticking time-bomb.
> Setup proper monitoring for Kafka & Zookeeper
> Store your metrics in Prometheus and visualise in Grafana

Operations for Apache Kafka

> Perform the most common and difficult operations demanded in your Apache Kafka Cluster
> Roll Restart your brokers in only one command
> Update Kafka Configurations safely with no downtime
> Rebalance Partitions and Change the Replication Factor of your topics
> Add, Replace and Remove Brokers.

Upgrades for Apache Kafka

> Understand the subtlety of upgrading a Kafka Cluster
> Step by step explanation on upgrading a Cluster with no Downtime

----------------------------

PRE-REQUISITES - THIS IS AN ADVANCED COURSE

> Good Understanding of Kafka & Kafka Setup
> Basic Understanding of AWS Cloud.
> Knowledge of how to SSH into a Server
> Good Understanding of Linux & computing overall
> Strong Ability to use the Command Line

-----------------------------

Instructors

Stephane Maarek is the instructor of this course. He is an Apache Kafka Expert, and the author of the highly-rated Apache Kafka Series on Udemy, having taught already to 40000+ students and received 12000+ reviews.  He shares all his Apache Kafka knowledge on the platform, taking the time to explain every concept and provide students with both theoretical and practical dimensions.

Hands On Based Course

Note: This course is advanced and little time will be spent on basics. It will incur cost in AWS

This course is the first and only available Kafka Monitoring and Operations on the web. Get it now to become an Apache Kafka expert!

Section outline:

  1. Course Introduction: Let's learn why we need Monitoring & Operations in Apache Kafka

  2. Kafka Quick Setup in AWS: Setup Kafka using AWS CloudFormation. No Knowledge of CloudFormation is required

  3. Kafka Administration Setup: Setup Basic Admin and Monitoring Tools on our Cluster

  4. Kafka Monitoring Setup: Grafana + Prometheus: Setup Kafka & Zookeeper Monitoring using Prometheus and Grafana

  5. Kafka Operations: Learn the most common and difficult Kafka operations 

  6. Kafka Cluster Upgrade: Learn the steps to properly upgrade a Kafka Cluster with no downtime

This Course Also Comes With:

✔ Lifetime Access to All Apache Kafka course Future Updates

✔ A responsive instructor in the Q&A Section

✔ Links to interesting articles, and lots of good code to base your next applications onto

✔ Udemy Certificate of Completion Ready for Download

✔ A 30 Day "No Questions Asked" Money Back Guarantee!

Learn Kafka Monitoring & Operations Now!

Kafka Monitoring and Operations are crucial knowledge for any Kafka Administrator. Get all the necessary knowledge with this Apache Kafka Series course! The job market will definitely need people like you

"Take This Course" risk free and learn the Kafka Monitoring & Operations now!

Who this course is for:
  • Kafka Administrators who want to learn how to setup monitoring
  • Kafka Administrators who want to learn how to perform Kafka operations
  • Kafka Administrators who want to learn how to upgrade a Kafka cluster
Course content
Expand all 64 lectures 05:08:07
+ Kafka Quick Setup in AWS
8 lectures 39:23
What is AWS CloudFormation
01:32
Target Architecture and Setup
04:18
Create AWS Account & Cost of Tutorial
00:22
[Hands-On] Deploy Zookeeper Using CloudFormation
08:15
[Hands-On] Deploy Kafka Using CloudFormation
07:38
[Hands-On] Overview of our Zookeeper Cluster
09:26
[Hands-On] Overview of our Kafka Cluster
05:03
[Hands-On] Stopping the Kafka Cluster
02:49
+ Kafka Administration Setup
6 lectures 34:12
[Hands-On] Overview and Setup of Monitoring Machine
08:10
IMPORTANT: Security
00:25
[Hands-On] Setup Zoonavigator
07:52
[Hands-On] Setup Kafka Manager
08:08
[Hands-On] Setup Kafka Monitor
09:27
Commands Cheat Sheet
00:10
+ Kafka Monitoring Setup: Grafana + Prometheus
16 lectures 01:17:47
Overview of Monitoring
05:06
[Exercise] Broker 2 & 3 Prometheus Setup
00:17
[Solution] Broker 2 & 3 Prometheus Setup
05:20
[Exercise] Zookeeper & Prometheus Setup
00:18
[Solution] Zookeeper & Prometheus Setup
09:40
[Hands-On] Setup Grafana on Administration Server
06:49
[Hands-On] Setup Kafka Dashboard on Grafana
05:43
[Hands-On] Adding Graphs to Grafana
10:15
[Exercise] Adding more graphs to Grafana
00:08
[Solution] Adding more graphs to Grafana
03:01
Find Grafana dashboard in the code
00:03
What about monitoring Kafka Connect, Kafka Streams, etc?
00:26
[Hands-On] Observing Grafana when we kill a broker
05:05
Considerations to make the setup production-ready
01:59
+ Kafka Operations
20 lectures 01:57:01
[Hands-On] Install Jolokia Agent on Broker 1
07:54
[Exercise] Install Jolokia on Broker 2 & 3
00:09
[Solution] Install Jolokia on Broker 2 & 3
03:38
[Hands-On] Install Rolling Restart Script
14:03
[Hands-On] Run Roll Restart Script
07:21
[Hands-On] Updating Configurations 1st method (rolling restart)
03:45
[Hands-On] Updating Configurations 2nd method (kafka-configs command)
11:18
[Hands-On] Rebalancing Partitions using plain commands and JSON
12:10
[Hands-On] Rebalancing Partitions using Kafka Manager
02:40
[Hands-On] Rebalancing Partitions using LinkedIn tools
08:10
[Hands-On] Increasing Replication Factor using plain commands and JSON
06:38
[Hands-On] Increasing Replication Factor using LinkedIn tools
04:47
[Hands-On] Adding Broker 4 using CloudFormation
09:55
[Exercise] Moving partitions over to Broker 4 using the CLI tools
00:08
[Solution] Moving partitions over to Broker 4 using the CLI tools
04:30
[Hands-On] Replacing a Broker keeping EBS Volume
07:02
[Hands-On] Replacing a Broker losing an EBS Volume
03:41
Removing a Broker Overview
01:00
[Exercise] Remove Broker 4
00:10
[Solution] Remove Broker 4
08:01
+ Kafka Cluster Upgrade
7 lectures 25:22
Upgrading a Kafka Cluster Overview
02:19
[Advanced] Zero Copy, Up and Down Conversions
08:06
[Hands-On] Kafka Broker Upgrade Step 1
04:02
[Hands-On] Kafka Broker Upgrade Step 2
02:58
[Hands-On] Kafka Broker Upgrade Step 3
03:03
[Hands-On] Kafka Broker Upgrade Step 4
01:40
[Hands-On] Kafka Broker Upgrade Step 5
03:14
+ Next Steps
4 lectures 05:05
Cleaning up AWS
01:17
Next Steps & Congratulations!
02:02
THANK YOU!
01:32
Bonus Lecture: Student Special Coupons for my Other courses
00:14