What is Zookeeper?

Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
A free video tutorial from Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
Best Selling Instructor, Kafka Guru, 9x AWS Certified
4.7 instructor rating • 37 courses • 574,934 students

Lecture description

Understand what is Zookeeper and Zookeeper Basics

Learn more from the full course

Apache Kafka Series - Kafka Cluster Setup & Administration

Hands-On Training on ZooKeeper Quorum Setup, Kafka Cluster Setup and Administration in AWS.

04:02:45 of on-demand video • Updated October 2020

  • Setup a Zookeeper and Kafka cluster on three machines in AWS
  • Learn how to deploy Kafka in Production and understand the target architecture for clusters in AWS
  • Setup ZooKeeper Cluster, learn its role for Kafka and usage
  • Setup Kafka in Cluster Mode with 3 brokers, including configuration, usage and maintenance
  • Shutdown and Recover Kafka brokers, to overcome the common Kafka broker problems
  • Configure Kafka Cluster with production settings and optimisations for better performances based on your workload
  • Setup web administration tools using Docker: ZooNavigator, Kafka Manager, Confluent Schema Registry, Confluent REST Proxy, Landoop Kafka Topics UI
  • Administer Kafka using Kafka Manager
English [Auto] So here's the fun. Here it actually begins. We're going to set up the zookeeper in a remote So it's going to be a multi server deployment and we're going to discuss configuration and et cetera et cetera. So before we get started and set up a zookeeper let's try to understand what is cheaper. So zookeeper is a pillar for so many distributed application because it provides features that are awesome. The first one is distributed configuration management so it can manage the configuration of distributed systems. It can also provide election and provide consensus. That means if if many servers ask who is who is the leader who is the chief and zookeeper's as you are. And that's it. You think that would be easy but it's really tough. So Deibert as just really well it also does coordination in Luxx that's more low level but it's good to know. And finally in Cafcass in Cafcass you know case it does key value store so it can store many configuration for topics for brokers et cetera et cetera. We're going to see this when we do a deep dive. So not just Cafcass but zookeeper's used by Hadoop and other Big Data Systems. It's an Apache project that's steller. It's very stable and it hasn't had any major release in many years because it's so stable. So if you're wondering hey what kind of version of zookeeper are we going to use the branch three point four has been stable for years and years. And that's the one we're going to use. You can say there's 3.5 as well. Should we use it. And 3.5 is about to be awesome. It has many features I can't wait to use but it's not ready yet. It's been in development for many many years and it's still in beta. So I do not recommend for you to use 3.5 zookeeper at all. OK. Kavkaz not ready for it and I'm not ready for it. And you're not ready for it. OK. So again for this tutorial we're going to use a cheaper 3.4 and that's going to work out just fine. Now let's just look at zookeeper is. So we all know what facets to look like. There is the slash right here and that means routes. That's the root of your file system. Like in Linux and then we could have folders in file systems and in zookeeper we can also the same concept where are you where are your roots as a node and we'll call that node app could be whatever was called app. And again if the app is a directory Well we can have multiple subdirectories or files. We're going to slash apps slept. Finance or a slash app slash sales. OK so zookeeper is really like a file system. OK there are a few differences but it looks like one and each of these things is a.. OK. So let's just talk about terminology a little bit. It has an internal storage structure like a tree. You see like on the left that's got a tree. We have the roots at the very top. And then you have branches and at the very bottom you have leaves or nodes. So each node is called a Z node and Y Z. Well because zookeeper with a zed. So each node is called Z-code. Each node has a path. So in the blue boxes for example in the middle one the path is slash app and in bottom it's slash absolute finance. OK so each node has a path. Each node can be persistence or ephemeral. So what's the difference. The persistence Z-code is something else stay alive all the time. It's set in stone zookeeper will remember it all the time an ephemeral Xeno's is as you know that will just go away. If your app disconnects so each of them have advantages and Carriker uses both not for you to worry about but it's good to know that both exist. Each node can store muscles you know. Or you can source all data and then finally you cannot rename it. It's impossible. You could copy some but you cannot rename them finally and that's super awesome and that's one of the best feature of zookeeper zookeeper Aurukun clients can watch Xeno's And when you watch as he knows you watch one for changes. So say my app finance changes I can watch it and when it changes zookeeper will let me know hey it has changed. You should check it out when I can get the updated value. And that's awesome. So zookeeper is very light and feature. A very very minimal project but what it does it does really really well. So that's a high level overview zookeeper. Kafka uses all the features that are described. OK not for you to worry about but it's really good of you to know about these things. OK so hopefully you have a better understanding of what zookeeper is will get you play with. Keep in the next lectures but hopefully that these two Sly's really help you out. All right so in the next lecture.