Apache Kafka Series - Kafka Cluster Setup & Administration
4.7 (75 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
764 students enrolled
Wishlisted Wishlist

Please confirm that you want to add Apache Kafka Series - Kafka Cluster Setup & Administration to your Wishlist.

Add to Wishlist

Apache Kafka Series - Kafka Cluster Setup & Administration

Hands-On Training on ZooKeeper Quorum Setup, Kafka Cluster Setup and Administration in AWS.
Best Selling
4.7 (75 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
764 students enrolled
Created by Stephane Maarek
Last updated 8/2017
English
Curiosity Sale
Current price: $10 Original price: $185 Discount: 95% off
30-Day Money-Back Guarantee
Includes:
  • 4.5 hours on-demand video
  • 3 Articles
  • 1 Supplemental Resource
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
What Will I Learn?
  • You will setup a Zookeeper and Kafka cluster on three machines
  • You will learn how to deploy Kafka in Production and understand the target architecture for clusters in AWS
  • You will learn what is ZooKeeper, its role for Kafka, setup and usage
  • Complete understanding of Kafka setup in cluster mode, including configuration, usage and maintenance
  • You will have a Kafka broker shutdown and recovery demonstration, to overcome the Kafka broker problems
  • You will learn Kafka production settings and optimisations for better performances based on your workload
  • You will learn how to setup web administration tools using Docker: ZooNavigator, Kafka Manager, Confluent Schema Registry, Confluent REST Proxy, Landoop Kafka Topics UI
  • You will have hands-on learning on Kafka common Administrative tasks using Kafka Manager
View Curriculum
Requirements
  • Strong knowledge on Computers, Servers and Networking
  • Strong knowledge of Linux commands, we will use Ubuntu 16.04
  • Good knowledge of AWS, we will use it as our cloud provider
  • Good knowledge of Apache Kafka
  • An AWS account and a budget to setup the required machines for the course
Description

Attention all Struggling System Administrators, Architects or even Developers. Here is course on how to setup and deploy Apache Kafka in Cluster mode.

Do you want to learn how to setup a Kafka Cluster on multiple servers and struggling with it?

Do you want to understand all the steps to perform a Kafka setup and Kafka Deployment in Production?

Have you been trying endlessly to deploy a Kafka Cluster, but haven't had any luck?

If you have answered YES, then you are at the right place…!!!

Learn from Stephane Maarek, the Best-Selling Instructor on Udemy in Apache Kafka and Big Data! 

This course is designed for the System Administrators, Architects or even for Developers. This course includes the step by step guide for installing a ZooKeeper Quorum and a Kafka Cluster. This course also provides hands-on training on Kafka Administration.

This course has Amazon Web Services for our cloud environment, which is used by many companies around the world! AWS Knowledge is a pre-requisite. 

In this course, we will cover what is ZooKeeper, its architecture, what is its role in Apache Kafka, and the setup, installation and configuration on multiple machines!

This Course reveals exactly how your Kafka Cluster on multiple machines should be setup and configured. We will start with understanding the Kafka basics, cluster size and the configuration. We will also have a hands-on learning on AWS Setup, Single Broker Setup, Multi Broker Setup, Testing the Cluster, Kafka Manager (Cluster Management), Demonstrating Kafka Resiliency etc. We will enlighten you on Kafka Performance with respect to I/Os, Network, RAM, CPU, OS (Operating System) etc. You will have hands on learning on how to run Kafka in Production on AWS, how to change a Kafka Broker Configuration and we will also cover the Advanced Kafka Configurations.

 

Why I should take this course?

  • With over 4 hours of videos and around 50 classes, you will get a great understanding of how to deploy Kafka in production and you will understand the target architecture for clusters in AWS
  • You will learn what is ZooKeeper? It’s role for Kafka, How to setup ZooKeeper? and Its usage
  • You will have a complete understanding Kafka setup including setup, configuration, usage and maintenance etc.
  • Inside the course you will have a Kafka broker shutdown and recovery demonstration, which will help you to understand how to overcome the Kafka broker problems
  • You will learn Kafka production settings and how to optimise settings for better performance
  • You will learn all the required tool setups such as ZooNavigator, Kafka Manager, Confluent Schema Registry, Confluent REST Proxy, Landoop Kafka Topics UI.
  • You will have hands-on learning on Kafka common Administrative tasks using Kafka Manager
  • You have life-time access to this course and a 30-day satisfaction guaranteed with this course

 

Overview of the Course Contents -

Section 1 – Course Introduction: In this section, we will have introduction to the course. We will talk about what are the course pre-requisites, course objectives, who are the targeted students for this course. You will also have course material for download in this section!

Section 2 - Target Architecture: In this section, we will see the cloud providers such as AWS and explore other alternatives for AWS.  We will also cover ZooKeeper Quorum Architecture and Kafka Cluster Architecture in this section.

Section 3 - Zookeeper Quorum Setup: In this sections, we will learn what is ZooKeeper, the Zookeeper role in Kafka Cluster, and ZooKeeper Configuration. We will have hands-on labs on AWS Setup, Zookeeper Single Machine Setup and Zookeeper Quorum Setup.

Section 4 - Kafka Cluster Setup: In this section, we will gain some practical hands-on experience on further AWS Setup, Single Kafka Broker Setup, Multi Kafka Broker Setup etc. Before that we will see Apache Kafka Basics on Kafka Configurations. We will have a demonstration on how to change the Kafka Broker configuration and we will also cover advanced Kafka configurations in this section.

 

After taking this course, you will become proficient in deploying Kafka in production! You will know all the required setups and installations and Kafka production settings and will be able to set-up Kafka in cluster mode in your own environment!

So, what are you thinking go ahead and click on “Enrol Now” button.

You have 30 days’ money back guarantee and a lifetime access to the course…!!! See you inside the course…!!!

Who is the target audience?
  • This course is designed for System Administrators or Architects, who would like to learn or setup a Kafka Cluster on multiple servers
  • This course is also ideal for Developers who want to understand the inner depth of a Kafka setup
  • This course is for expert students and should not be taken until all the pre-requisites are met
Students Who Viewed This Course Also Viewed
Curriculum For This Course
55 Lectures
04:17:48
+
Course Introduction and Pre Requisites
8 Lectures 16:20

Go over the course objectives for this course

Preview 03:28

Description of what this course will not go over

Preview 02:58

Pre requisites description for the course

Preview 03:03

Understand who this course is for

Preview 01:09

Learn more about your instructor, Stephane Maarek!

Preview 00:54

Description of how to ask questions in the Q&A

How to ask questions in the Q&A
02:26

Five very important course rules we have to go over

5 Golden Course Rules
02:12

Downloading the course material
00:10
+
Target Architecture
6 Lectures 11:15
Cost of the tutorial
00:42

Understand the reasons behind choosing AWS for this course and talking about alternatives

Cloud Provider: AWS & Alternatives
02:15

Go over the final target Zookeeper architecture

Preview 01:05

Go over the final target Kafka architecture

Preview 02:43

Go over the final architecture, including the web tools

Preview 01:10

Note on IP addresses and DNS
03:20
+
Zookeeper Quorum Setup
17 Lectures 01:47:15

Understand what is Zookeeper and Zookeeper Basics

What is Zookeeper?
04:55

Analyse the role Zookeeper has with regards to Kafka

Zookeeper Role in a Kafka Cluster
02:49

Learn how to size a Zookeeper Quorum. 

Zookeeper Quorum Sizing
05:32

Learn about Zookeeper Configuration

Zookeeper Configuration
03:14

Start configuring our AWS (Amazon Web Services) environment by creating the security groups, and our first EC2 instance

Preview 11:25

Learn how to SSH into our machines. Use Putty if you have a windows machine

How to SSH Into our Servers?
04:18

In this lecture we will setup one machine with Zookeeper on it 

Hands-On: Single Zookeeper Machine Setup
12:51

Learn how to setup Zookeeper as a Service on your AWS EC2 machine

Hands-On: Zookeeper Service Setup
06:22

Learn how to use the Zookeeper Command Line Interface to interact easily with Zookeeper

Using the Zookeeper Command Line Interface (CLI)
08:13

Create two new EC2 instances from a custom AWS AMI

Hands-On: Zookeeper Quorum setup - Part 1 AWS
08:03

Setup the other two Zookeeper Machines and make sure the quorum is working

Hands-On: Zookeeper Quorum Setup - Part 2 Zookeeper
15:07

Learn about Zookeeper Four Letter Words (4LW) to diagnose issues with Zookeeper

Zookeeper Four Letter Words
04:52

Learn and understand the files created by Zookeeper

Zookeeper Internal File System
01:59

Learn the various factors that can impact heavily Zookeeper's performance

Factors impacting Zookeeper Peformance
01:58

Advice on how to run Zookeeper in AWS

Note: Zookeeper in AWS
01:48

Setup a new AWS EC2 Machine and install Docker on it, to be able to install our Web tools

Hands-On: Web tools AWS machine Docker setup
06:20

Learn about all the various options to manage Zookeeper, and install Zoonavigator to start visualising our Zookeeper cluster

Management Tools for Zookeeper & Zoonavigator demo
07:29
+
Kafka Cluster Setup
22 Lectures 02:00:49

Learn the basics of Apache Kafka

Kafka Basics
01:49

Understand how to choose the size of a Kafka cluster

Preview 04:26

Overview of Kafka Configuration options

Kafka Configuration
08:27

Walk through of the required AWS Network and instance changes to setup Kafka

Hands-On: Kafka AWS Setup
11:29

Kafka Single Broker Setup, on one instance. We will learn how to start Kafka

Hands-On: Single Kafka Broker Setup
12:06

Learn how to run Kafka Commands against a Kafka Broker

Hands-On: Running Kafka Commands
05:11

Setup two additional Kafka Brokers to finalise our cluster setup

Hands-On: Kafka Multi Broker (Cluster) Setup
10:52

Learn how to test the Kafka Cluster using various Kafka commands

Hands-On: Testing the Kafka Cluster
05:37

Understand how networking works in Kafka and where you can connect to in Kafka

Can I connect to my Kafka cluster?
03:28

Learn about the most important Kafka Setting, advertised.listeners, and how it impacts networking

advertised.listeners setting - most important setting
06:11

Install and start using Kafka Manager for Kafka Cluster Management

Preview 10:29

Demonstrations of Kafka Resiliency by stopping brokers in order, and showing how Kafka Manager and the cluster health changes

Hands-On: Demonstrating Kafka Resiliency
14:25

Learn how IO impacts Kafka Performance

Kafka Performance: I/O
01:47

Learn how Network impacts Kafka Performance

Kafka Performance: Network
02:05

Learn how RAM (memory) impacts Kafka performance

Kafka Performance: RAM
01:19

Learn how CPU impacts Kafka Performance

Kafka Performance: CPU
01:34

Learn how your OS can impact Kafka Performance

Kafka Performance: OS (Operating System)
01:27

Other notable things to look out with regards to Kafka performance

Kafka Performance: Other
01:10

Learn the subtleties of Running Kafka in production on AWS

Running Kafka in Production on AWS
01:36

Hands on lecture to learn how to change a Kafka configuration

Hands-On: How to change a Kafka Broker Configuration
04:01

Learn about the advanced Kafka configuration parameters, that you may want to tweak in production for your cluster

Advanced Kafka Configuration
05:43

Kafka Topics UI Setup (alongside Confluent REST Proxy and Confluent Schema Registry)

Landoop Kafka Topics UI, Confluent REST Proxy, Confluent Schema Registry Setup
05:37
+
Next steps!
2 Lectures 02:17

Learn about the next steps in your Kafka in production learning adventure

What's Next? Learning further
01:19

Special discounts for my other courses!!

Bonus Lecture - Coupons For My Other Courses - Student Special Offer
00:58
About the Instructor
Stephane Maarek
4.6 Average rating
1,164 Reviews
5,357 Students
6 Courses
Best Selling Instructor, Kafka Guru, AWS Solutions Architect

Stephane is a solutions architect, consultant and software developer that has a particular interest in all things related to big data and analytics. He's also a best seller instructor on Udemy for his courses in Apache Kafka, Apache NiFi and AWS Lambda!

He loves Apache Kafka. He regularly contributes to the Apache Kafka project and wrote a guest blog post featured on the Confluent website, the company behind Apache Kafka. 

He also is an AWS certified Solutions Architect, and has many years of experience with technologies such as Apache Kafka, Apache NiFi, Apache Spark, Hadoop, PostgreSQL, Tableau, Spotfire, Docker and Ansible amongst many others. 

His favourite programming languages are Scala and Python, and he plans on learning Go soon. 

During his spare time he enjoys cooking, practicing yoga, surfing, watching TV shows, and traveling to awesome destinations!