Centralized Logging and Monitoring with Kubernetes

Kubernetes is an open source platform designed to automate deployment, scaling, and operation of application containers. Kubernetes automates various aspects of application development, which is extremely beneficial for enterprises. Centralized logging is crucial for any production-grade infrastructure, especially in a containerized architecture. Since Kubernetes is dynamic and does not store change logs except the recent changes, logging and monitoring is highly imperative for saving pod logs.

In this course, you’ll learn to analyze and locate critical pod log files in your Kubernetes clusters. You’ll create a centralized logging system with a configured EFK (Elasticsearch, Fluentd, and Kibana) stack for Kubernetes. Using a hands-on approach, you’ll follow the entire logging and monitoring process, which actually goes hand-in-hand. In your Kubernetes cluster, you’ll find out that your clusters are working with too many containers and it’s difficult to keep track of each of them.

You’ll learn how to build your centralized logging and send data for monitoring. To set up centralized logging, you’ll establish one logging agent per Kubernetes node to collect all logs of all running containers from disk and transmit them to Elasticsearch. You’ll search for log data, monitor the containers, and also collect metrics using Kibana. You’ll decide how your final log data will be presented. By the end of the course, you’ll be able to use centralized logging and monitoring techniques for debugging purposes to find out reasons for crashes, and trigger alerts if there is a spike in error messages (which can be more efficient than a system health check).

About the Author

Walter Dolce is a Software and Platform Engineer based in London, United Kingdom. He has worked for both small and medium-sized businesses as well as large enterprises such as the BBC and Just Eat. Over the course of the years, he has developed a deep knowledge of various areas of Software Engineering concepts and practices such as test-driven development, behavior-driven development, SOLID principles, design patterns, and more. He later transitioned to the DevOps/Platform Engineering space where he used that knowledge to implement highly available, resilient systems and platforms running on today’s major Cloud providers such as Amazon Web Services, Google Cloud Platform, and Microsoft Azure.