What is Zookeeper?

Stephane Maarek | AWS Certified Cloud Practitioner,Solutions Architect,Developer
A free video tutorial from Stephane Maarek | AWS Certified Cloud Practitioner,Solutions Architect,Developer
Best Selling Instructor, Kafka Guru, 9x AWS Certified
4.7 instructor rating • 41 courses • 929,058 students

Lecture description

Understand what is Zookeeper and Zookeeper Basics

Learn more from the full course

Apache Kafka Series - Kafka Cluster Setup & Administration

Hands-On Training on ZooKeeper Quorum Setup, Kafka Cluster Setup and Administration in AWS.

04:02:45 of on-demand video • Updated July 2021

  • Setup a Zookeeper and Kafka cluster on three machines in AWS
  • Learn how to deploy Kafka in Production and understand the target architecture for clusters in AWS
  • Setup ZooKeeper Cluster, learn its role for Kafka and usage
  • Setup Kafka in Cluster Mode with 3 brokers, including configuration, usage and maintenance
  • Shutdown and Recover Kafka brokers, to overcome the common Kafka broker problems
  • Configure Kafka Cluster with production settings and optimisations for better performances based on your workload
  • Setup web administration tools using Docker: ZooNavigator, Kafka Manager, Confluent Schema Registry, Confluent REST Proxy, Landoop Kafka Topics UI
  • Administer Kafka using Kafka Manager
English [Auto] So here's the fun here, it actually begins, we're going to set up zookeeper in a remote. So it's going to be a server deployment and we're going to discuss configuration and et cetera, et cetera. So before we get started and set up, zookeeper, let's try to understand what is zookeeper. So zookeeper is a pillar for so many distributed application because it provides features are awesome. The first one is distributed configuration management so it can manage the configuration of distributed systems. It can also provide election and provide consensus. That means if if minister was asked who was the leader, who is the chief and zookeepers as you are and that's it, you think that be easy, but it's really tough. So Zookeeper does this really well. It also does coordination in luks the small low level, but it's good to know. And finally in Kavkaz, in Cafcass case, it does key value store so he can store many configuration for topics, for brokers', et cetera, et cetera, etc.. We're going to see this when we do a deep dive. So not just Kafka, but zookeepers used by Hadoop and other big data systems. It's an Apache project that's stellar. It's very stable and it hasn't had any major release in many years because it's so stable. So if you're wondering, hey, what kind of version of Zookeeper are we going to use? The branch three point four has been stable for years and years, and that's the one we're going to use. You can say, hey, there's three point five as well, should we use it? And three point five is about to be awesome. It has many features there can't wait to use, but it's not ready yet. It's been in development for many, many years and it's still in beta. So I do not recommend for you to use three point five Zookeeper at all. OK, Kafka is not ready for it and I'm not ready for it. And you're not ready for it. OK, so again, for this tutorial, we're going to use Zookeeper three point four and that's going to work out just fine. Now, let's just look at what zookeeper is so we all know what filesystem look like, there is this right here and that means roots. That's the root of your filesystem, like in Linux. And then we could have folder's and filesystems. And in Zookeeper, you can also have the same concept where you where your roots has a node and we'll call that node EBP. It could be whatever, but we'll just call app. And again, if the app is a directory, well, we can have multiple subdirectories or files. We can have apperception, finance or app slash cells. OK, so zookeeper is really like a filesystem. OK, there are a few differences, but it looks like one and each of these things is a node. OK, so let's just talk about terminology a little bit. It has an internal structure like a tree. You see like on the left that's called a tree. We have the roots at the very top and then you have branches and at the very bottom you have leaves or nodes. So each node is called the node. And why? Well, because zookeeper with a Z, so each noticeable node, each node has a path. So in the blue boxes, for example, in the middle one, the path is up and in bottom. It's about finance. OK, so each zip code has a path. Each the nodes can be persistance are ephemeral. So what's the difference? The persistency node is something they'll stay alive all the time. It's set in stone. Zookeeper will remember it all the time. An ephemeral xenos is a zero that will just go away if your app disconnects. So each of them have advantages. OK, and Kafka uses both not for you to worry about, but it's good to know that both exists. Each node can store multiple zeros or you can source also data. And then finally, you cannot rename, you know, it's impossible, you can copy some, but you cannot rename them finally and that's super awesome. And that's one of the best feature of zookeeper zookeeper can clients can watch Xenos. And when you watch, as you know, you watch one for changes. So say my app finance changes, I can watch it. And when it changes, Zookeeper will let me know. Hey, it has changed. You should check it out and I can get the updated value and that's awesome. So Zookeeper is very light in features. It's a very, very minimal project. But what it does, it does really, really well. So that's a high level overview of Zookeeper. KFKA uses all the features that are described. OK, not for you to worry about, but it's really good for you to know about these things. OK, so hopefully you have a better understanding of what Zookeeper is. We'll get you play with Zookeeper in the next lectures, but hopefully that these two slides really help you out. All right. See you in the next lecture.