
Meet a seasoned quality engineering leader with 20 years of industry experience. Explore performance testing with AI using JMeter, Lighthouse, and K6, and apply agile, cloud, and machine learning practices.
A brief understanding of the project and tools that are used in this lecture
Understand the trends in software quality engineering with 3 powerful resources:
1. State of Devops
2. World Quality Report
3. Continuous Testing Report
Understand what are performance validations
A small discussion as to that are the goals and objectives of performance testing
Here I will showcase 2 examples as to why performance testing is needed and what is the consequence of not having proper performance validations prior to any release.
Test the performance of network, page load, printing, boot time, cryptography, web services, and encoding/decoding. Choose what to test based on project context and industry.
Start performance testing with unknown user counts by referencing industry benchmarks like 2.5–3 seconds page load. Compare targets to top competitors and assess login and responsiveness using common sense.
Identify target app and hosting environment, select a test tool, and define test scenarios. Enable monitoring and debugging with Azure insights, and ensure infrastructure and test data support load testing.
Measure cpu and ram during infrastructure performance tests and know when to stop; distinguish cpu-intensive from memory-intensive workloads and use 90% cpu or 95% memory thresholds.
https://jmeter.apache.org
https://github.com/nosqlbench/nosqlbench/ | https://docs.nosqlbench.io
https://k6.io
https://www.hammerdb.com
https://pagespeed.web.dev
Simulate user traffic with tools like k6 and loadrunner to test APIs, front-end, back-end, and databases, measuring response times and resource usage across loads.
Visualize the performance test pyramid guiding testing from low-level functions to APIs and end-to-end scenarios, including scalability considerations, with the bulk of tests close to development for speed and feedback.
Explore how load testing benchmarks system performance under a constant load, using ramp up, steady load, and ramp down to identify bottlenecks and optimize with 100 to 250 users.
Perform a short K6 load test from Grafana to simulate 1200 users over 30 seconds, capture http 200 checks, 80 ms average latency, and observe CPU and memory trends.
Perform stress testing by pushing beyond the steady state to reveal the system's limits, evaluate response time, memory, and CPU behavior, and validate graceful shutdown, throttling, and chaos engineering.
Combine acceptance testing with performance testing using BDD, tie in real user analytics, and let AI map behavior into prioritized given-when-then test cases for end-to-end performance.
Explore an on-prem performance testing setup with three Windows VMs on a VMware layer and container orchestration, contrasting cloud-hosted apps with a local load generator for isolation and cost insights.
Explore cloud-based load testing with SaaS providers, where you simulate traffic from multiple geographies on hosted infrastructure you don’t manage, paying only for used virtual users via a simple GUI.
Explore K6 browser testing as a browser-based, headless load testing tool, compatible with the Playwright API, for quick smoke tests and experiments in your performance testing workflow.
Run your first automated performance test with k6 by creating a test.js script, launching it with ten virtual users for 30 seconds, and reviewing results.
Assess the performance test report by reading data received, data sent, and average response time. Clarify iteration counts, virtual users, and percentiles (90th, 95th) to evaluate server performance.
Test application 1: https://test-api.k6.io/
Test application 2: https://test.k6.io/
Understand the differences between scenarios, executor and stages in K6 script definition
Explore how performance testing contrasts requests per second with virtual users, compares synchronous and asynchronous load, and shows how median and average reveal response-time distributions.
Interpret performance results visually by plotting load versus response time and filtering the csv for readability. Prefer Grafana Cloud reporting, or, if needed, use ChatGPT as a last resort.
Run your performance tests in the cloud from VS Code using k6 cloud, configure environment variables with your API key, and monitor results in Grafana Cloud.
Demonstrates running k6 tests from GitHub into Grafana Cloud, using a Grafana API token in GitHub secrets, and comparing CI/CD results with in-cloud performance dashboards.
Explore how Google Lighthouse measures loading performance from the user’s perspective across devices, and access it via pagespeed.web.dev or browser extensions.
Run a Google Lighthouse assessment to measure performance, accessibility, best practices, and SEO, loading the page multiple times to produce final results and check mobile-first behavior.
Explore how Google Lighthouse computes a performance score by weighting metrics such as first contentful paint, largest contentful paint, total blocking time, cumulative layout shift, and speed index.
Learn to interpret a Google Lighthouse report by examining the tree map and coverage to identify unused JavaScript and speed up pages. Optimize for mobile, accessibility, and SEO.
Don't obsess over performance scores from tools like Google Lighthouse, PageSpeed, Yellow Labs, and GTmetrix; they vary. If the app feels fast, prioritize user experience and SEO over metrics.
Learn how throttling affects performance testing by disabling or monitoring it to avoid false results, verify correct handling of denied requests, and compare test scenarios.
Explore how caching layers—from browser to database, content delivery networks, and memory—boost performance, and why testing with caching disabled first then gradually enabling layers provides realistic measurements.
Explore how sticky sessions affect performance in distributed clusters, explaining session replication, load balancers, and why disabling sticky sessions can improve testing outcomes.
Learn how logging levels from tracing to fatal affect disk space, CPU, and network during performance testing, and when to use minimal logs in production.
This is a basic material on how to execute performance testing. I will cover the following aspects:
1. Understand what is performance testing and where does it fit into Software.
2. Understand why performance matters and how it impacts sales.
3. Examples where performance has broken a company.
4. What are the preconditions before running performance testing.
5. Shift Left on performance and feature testing environments.
6. Infrastructure configuration factors that influence performance test results.
7. Performance Testing infrastructure in the form of local, cloud or cloud only.
8. What to we monitor in performance testing and why does it matter
9 Practical example and tutorial on how to configure and use Grafana/K6 for local, remote and cloud testing.
10. Performance testing blocking pipelines with K6/Jmeter/Lighthouse and GitHub.
11. Approach to performance testing with live demos for Load/Spike and Stress Testing.
Laboratory that will demo how to create a performance testing framework with 0 knowledge of how Jmeter operates, by leveraging Generative AI - Chat GPT
Creating a Test Plan: You'll start by understanding the fundamentals of JMeter and how to create an effective test plan. We'll cover the key components and configurations needed to structure your tests accurately.
Adding Users: Learn how to simulate real user behavior by adding users to your test scenarios. We'll explore various user profiles and demonstrate how to configure JMeter for different user loads.
Adding Test Data: Discover the importance of realistic test data and how to incorporate it into your performance tests. We'll show you how to efficiently manage and manipulate data for thorough testing.
Adding Randomness: Understand the significance of randomness in performance testing and how to introduce it into your test scenarios. You'll learn techniques to create dynamic and unpredictable test conditions.
Interpreting Results: Dive into result analysis and learn how to extract valuable insights from your test runs. We'll cover metrics, graphs, and best practices for interpreting JMeter results effectively.
Integration with CI/CD and GitHub: Take your performance testing framework to the next level by integrating it seamlessly with your CI/CD pipeline and GitHub. Automate your testing process and ensure continuous performance monitoring.
By the end of this course, you'll have a basic grasp of building a basic JMeter performance testing framework and integrating it into your CI/CD workflow. You'll be equipped with the skills needed to continue your journey to meaningful performance tests, identify bottlenecks, and optimize your applications for peak performance.