The Complete End-to-End Logging Course

Learn what logs are and why they are useful for back-end visibility, including capturing exceptions, keep-alive signals, and analytics, with guidance on filtering by log levels.

Explore log levels, including debug, information, warning, error, and critical, and learn how to apply each level's purpose and conventions for end-to-end logging.

build a server-side stock price app that retrieves stock prices on user request and learn end-to-end logging from docker to grafana using gelf logs, logstash, mysql, and alerts.

Create a basic server API using the .NET Web API template with explicit controllers and logging logic. Run the app and access localhost/weather forecast to view five temperature responses.

Obtain an ILogger from the DI container in program.cs via app.services.getService and log that the application has started before app.run, noting its blocking behavior.

Learn how to change log levels with ILogger, write information and debug logs, and configure app settings to control which levels print without restarting, preparing for nlog integration.

Learn to write machine-parseable logs in JSON by creating a LogMessage class, applying dependency injection, and wiring a StockFinder controller that retrieves prices from Alpha Advantage.

Create a custom iLogger that outputs log messages as JSON by wrapping a log message object and leveraging a generic interface for dependency injection.

Refine nlog configuration to show only json logs, implement a memory cache via the decorator pattern for stock price retrieval, and integrate with dotnet di and the scruter package.

Wrap your code in a docker container by creating a dockerfile and dockerignore, build a lean image, map ports to the host, and view logs via stdout.

Create a Docker Compose file to run multiple containers and connect them, and learn how Docker Compose builds and runs the app with the image, showing logs in the terminal.

Explore how Docker logs work by shipping container stdout to the host and to a GELF destination, without changing the software, using Docker Compose and log configuration.

Configure logstash to receive GELP input on UDP port 12201, map this UDP port in docker-compose, and run the application alongside logstash to stream logs to the console.

Add a filter in Logstash to parse the message as JSON, enabling per-attribute handling, and test with HTTP input and curl to verify keys appear in the stream.

Refine Logstash filters by using prune and whitelist JSON field names, align variables with explicit JSON names, and test changes with curl to view only the selected keys.

Add categories to log messages to identify data like stock price and response time, using a category parameter and enum strings for easy parsing in logstash.

Install mysql under docker, add a persistent volume to the logstash docker compose, run it, then connect to the docker terminal to run sql commands as the database comes up.

Create a stock data database with tables for current prices, all historical prices, and response times, using ticket name and date fields, with ticks for time measures.

Create two MySQL store procedures to insert stock price data and log response time, separating logic for easy future changes, then install the SQL plugin for Logstash and configure output.

In this lecture, learn to extend your database project by enabling logstash to output sql, install the sql plugin, create a docker image with mysql connector, and update docker compose.

Configure Logstash to output to a SQL database with conditional outputs for stock price and response time, then run via Docker Compose and verify data in tables for Grafana visualization.

Add Grafana as a service to the logging Docker Compose, expose port 3000, and mount storage to enable dashboards, tables, graphs, and alerts from multiple data sources after admin/admin login.

connect grafana to a mysql data source by configuring hostname, port 3306, database stockdata, and root credentials, then save and test to confirm the connection.

Build a Grafana dashboard with a table visualization of stock prices, select data source and columns, apply a time filter, and add a panel showing response time by date.

Create an alert for high response time in a Grafana panel, configure last as the reduction, set evaluation every 10 seconds, and test firing with thresholds and email notifications.