
Explore big data visualization for games with Elastic Stack, generating and visualizing logs from sessions, builds, performance, and location data using Elasticsearch, Logstash, and Kibana.
Discover how to use this course through practical workshops. Join hands-on exercises with Q&A support and a setup for your working environment using free tools.
Learn to set up a local elastic stack by downloading, extracting, and configuring Elasticsearch, Logstash, and Kibana, then launch and verify connectivity via http on ports 9200 and 5601.
Set up python, vscode, and Windows Subsystem for Linux to prepare elastic stack data visualization and install the Elasticsearch package.
Set up Unreal Engine five and the Stack Robot sample project to collect game logs for big data visualization with Elastic Stack.
Explore the Elastic Stack basics by launching Elasticsearch and Kibana, importing sample flight data, and visualizing logs in dashboards using Kibana Discovery, Kibana Query Language, and Logstash.
Configure logstash to move json logs from local files into elasticsearch, using input, filter, and output blocks, and visualize them in kibana with dummy data and index management.
Learn to pass test log data to Elasticsearch with Python as an alternative to Logstash, including a Python script to index, update by id, and delete documents.
Visualize 500 dummy json logs with the elastic stack to build dashboards in elasticsearch, kibana, and logstash, including faction pie, playtime vs level line chart, and PvP rating table.
Visualize game session logs with Elastic Stack, tracking who played, device, build version, session duration, and hardware details to analyze crashes, performance, QA testing, and feature usage.
Generate simplified Unreal Engine 5 game session logs in json via blueprints, capturing session id, player name, start and end times, duration, build version, platform, and iso-formatted timestamps for Elasticsearch.
Learn to stream Unreal Engine session logs through Logstash to Elasticsearch and visualize them in Kibana, including encoding, date mappings, and a dedicated Unreal Engine session index and data view.
Visualize game session logs with Elastic Stack to explore build and platform usage, playtime, and hardware stats using a prepackaged Python data set and Kibana dashboards.
Explore location-specific game data visualization on a map by building two visualizations: player movement paths and event hotspots, using Unreal Engine data, Python tiles, and Kibana geo points.
Learn to build a tiled game world map in Kibana using Elastic Stack by rendering top-view textures, tiling them, serving locally, and configuring a tile map service.
Create a new location index with geo point fields in Kibana and route Logstash logs by path to location or session indexes. Use Unreal Engine blueprints to generate dummy data.
Generate and visualize Unreal Engine player location logs with Elastic Stack in part 1, retrieve and convert x and y to longitude and latitude, then map events in real time.
Generate and visualize a player's movement path by emitting two geo points (origin and destination) every few seconds and rendering a Kibana line with session ID and player name.
Leverage big data visualization to automatically analyze game performance, tracking fps and function times across hardware and builds to reveal trends and guide optimization.
Implement a minimal unreal engine 5 blueprint to generate performance logs, recording fps values and session metadata (player name, build version, platform, session id) with random activity types for visualization.
Compute the average fps from raw fps values with a logstash ruby filter and store it in the Unreal Engine Stackable Performance index, using runtime fields.
Create and compare an Elasticsearch runtime field to compute average fps at query time, using a data view in Kibana discovery and mirroring the Logstash calculation.
Extend the Kibana dashboard for game performance by adding average FPS visualizations—build version, platform, and activity type—and refine filters and alerts using Elastic Stack tools.
Explore big data visualization for games using the Elastic Stack, expanding to production level with cloud deployment, real session location, and performance metrics in the standalone build.
Welcome to Big Data Visualization for Games using Elastic Stack!
This course is your gateway to mastering data-driven insights for game development using the Elastic Stack (ELK).
Whether you're a Data Analyst, QA Engineer, Tech Lead, Pipeline Architect, Automation/DevOps Engineer, or a Tech Artist, this course is designed to equip you with the practical skills to process, analyze, and visualize game data for improved development workflows and decision-making.
What You’ll Learn
Throughout the course, you'll explore and implement Big Data visualization solutions, covering three essential types of game development logs:
Game Session Data: Track who played the game, for how long, and on which platform, providing insights and foundation for more specific metrics, like Crash-per-hour rate, average play session durations, etc.
Performance Data: Analyze historical Performance metrics (FPS, CPU/GPU usage, memory consumption, function execution times) across different builds, platforms, and gameplay scenarios to make informed decisions in performance optimizations.
Location-Specific Data: Recreate player movement path, map game crashes, rare boss kills, FPS dropped, and other key events using interactive game maps in Kibana.
By the end of this course, you’ll have a fully functional Big Data dashboard that transforms raw logs into actionable insights!
This course is fully practical (similar to my Python-related courses) where most of the time you're attending workshops with various challenges rather just watching raw-slides lectures.
As a source of our game logs throughout the course we will be using Unreal Engine 5 with its Sample Project Stack-O-Bot to mimic the real-world data and meaningful metrics for analysis.
All the tools involved in the course content have Free access.
Source Code included.