Apache Kafka Series - Kafka Security | SSL SASL Kerberos ACL

Set up a Kafka broker and ZooKeeper on an Amazon EC2 instance with an elastic IP, access it via SSH, and validate the setup with a simple producer and consumer.

Launch and secure an AWS EC2 Ubuntu instance, assign an Elastic IP, restrict SSH to your IP, create and download a key pair, and verify SSH access.

Install and run ZooKeeper and Kafka on two EC2 instances, download Kafka binaries, start ZooKeeper and Kafka as daemons, and verify both services are up using logs and ZooKeeper checks.

Set up systemd service units for Zookeeper and Kafka, enable them, and test start, stop, and restart while verifying status with systemctl and journalctl.

Test producer and consumer connectivity on a plaintext, unsecure Kafka setup by extending security groups, configuring the broker and ZooKeeper, and running local clients to verify messaging.

Set up a Kafka broker on two instances with a public DNS and elastic IP, created topics, and tested producer and consumer; configured security groups for local access.

Explore why ssl encryption (tls) is essential for secure, in transit data between Kafka and clients, and how encrypted packets prevent routers from peeking.

Discover how TLS/SSL encrypts client-server communications and how Kafka uses a private certificate authority with keystore and truststore to enable one-way and two-way SSL.

Learn ssl encryption in kafka by setting up a certificate authority, signing broker certificates, configuring key stores and trust stores, and testing with ssl producer and consumer on port 9093.

Create a local certificate authority with OpenSSL, generating a 4096-bit RSA private key and a public certificate with a one-year validity for signing self-signed Kafka SSL certificates.

Set up Kafka ssl by creating keystore and truststore, signing certificates, configuring broker properties for ssl on port 1993, restarting, and verifying with an ssl client connection.

Learn to configure Kafka clients for ssl encryption by creating a trust store from the certificate authority, setting client properties, and testing the producer and consumer over the ssl endpoint.

Enabling ssl in kafka adds a performance impact on brokers and clients, losing zero copy but increasing latency; Java nine diminishes the impact, while encryption and security remain.

Learn how SSL encryption is set up across Kafka and what happens behind the scenes, enabling you to diagnose issues and understand what is right.

discover ssl authentication in kafka, enabling mutual certificate checks between broker and client using a shared certificate authority, key stores, and acl-based access control.

Implement ssl authentication for Kafka by creating a client keystore, signing certificates, configuring broker ssl.client to required, and testing with producer and consumer.

Explore how sasl in Kafka enables Kerberos authentication and how it pairs with tls for encryption, using plain and scram protocols.

Explore Kerberos, an authentication protocol using a key distribution center, tickets, and service principals to securely connect clients to a Kafka service.

Provision EC2 instances and install MIT Kerberos to enable Kerberos-based authentication for Kafka. Configure the KDC, ACL, and admin principal, then start the Kerberos services.

Create user and service principals and export them to keytab files for Kerberos authentication. Install Kerberos clients and verify tickets with kinit and klist on both local and Kafka servers.

Configure Kerberos authentication in Kafka by updating broker config, keytab, and systemd env, then restart and verify startup. Update AWS security groups to allow port 1994 and Kerberos related ports.

Configure Kerberos authenticated Kafka clients with SSL and SASL, using a ticket cache. Produce and consume messages with console tools, validating tickets and simulating login failure when cache is empty.

Define access control lists to authorize Kafka clients for read/write on topics, consumer groups, and clusters. ACLs live in Zookeeper, require restricted admin access, and establish a super user role.

Enables and demonstrates Kafka authorization with acl rules, creating topic and user permissions (reader, writer, admin) and validating access via Kerberos-secured producer and consumer.

Explore zookeeper security fundamentals by enforcing authentication and authorization with zookeeper ACL, digest, and Kerberos, safeguarding Kafka metadata from unauthorized access.

Create ZooKeeper Kerberos principals, generate and export keytabs, copy them to Kafka nodes, and test ticket retrieval to validate ZooKeeper security.

Configure Kerberos authentication in Zookeeper by extending config files, creating a keytab and service principal, updating the startup script with Java security login config, and verifying authentication with Kafka.

Examine how zookeeper uses set nodes and default world read writable ACLs, compare authenticated versus unauthenticated access, and review Kafka topics metadata ACLs and Kerberos limitations.

Enable Zookeeper authorization on newly created topics by setting zookeeper.set.acl, strip the host and real parts from the Kerberos principal, then restart Kafka and Zookeeper and verify ACLs.

Shows recovering from accidental Zookeeper ACL changes by enabling a superuser via digest authentication. Use the Zookeeper shell to set proper ACL on the topic set node.

Transform non-secure ZooKeeper ACLs to secure ones with the ZooKeeper security migration tool, enabling authentication, authorization, and ACL enforcement across Kafka topics.

Explore securing a Kafka cluster with SSL for broker and broker-to-zookeeper connections, and using Kerberos for inter-broker authentication, while weighing network security groups as a simple option.

Complete this challenging Kafka security course by internalizing lessons, persevering through setup frustrations, and applying learnings to your organization. Share your certificate on LinkedIn and leave a five-star review.

Encourage learners to leave reviews and feedback after completing the course, noting the Udemy.com pop-up star rating, optional comments, and the bonus lecture with coupons for affordable education.