ISTQB Testing with Generative AI (CT-GenAI)

Examine generative AI foundations and its use in software testing, covering symbolic AI, classical ML, deep learning, large language models, multimodal models, and AI-powered test tasks.

Explore generative AI foundations and key concepts, including lm and slm, and differences in artificial intelligent systems. Learn how to leverage generative AI in software testing with core principles.

Define GenAI as a branch of artificial intelligence using large, pre-trained language models, trained in the cloud, to generate human-like text, images, or code.

Explore how tokenization breaks text into tokens for generative ai, showing how words, spaces, and characters become tokens and how each token carries history for processing.

Explore context windows and token limits that shape what AI remembers in long conversations. Learn to manage memory by starting new chats or summarizing prior dialogue.

Learn how multimodal models process text, images, audio, and video, and how they differ from single-modality models. Discover how these models can generate data types and handle interactions across modalities.

Discover how generative AI and large language models enhance ISTQB AI tester workflows by generating test cases, test scripts, and synthetic data, while reviewing acceptance criteria and analyzing defect patterns.

Explore the ai spectrum from symbolic ai and rule-based systems to classical machine learning and deep learning, highlighting data preparation, feature selection, model training, and pre-trained models for software testing.

Compare small language models and large language models, noting SLMs have fewer parameters and lower latency, suited for software testing content, while LLMs handle tasks but incur higher hallucination risk.

Explore tokenization and embeddings, where tokens become high-dimensional vectors encoding semantic, syntactic, and contextual relationships for language models.

Explore how transformer models transform input into output, handle long text, learn token relationships, and generate statistically plausible but sometimes hallucinated answers.

Understand non-deterministic behavior in AI systems by examining their probabilistic outputs, how input variability, temperature, and context windows produce different answers, and implications for software testing and test automation.

Practice tokenization with a tokenizer to split text into tokens and count them, noting how prompts and emojis impact token costs. Apply prompting and caching approaches to testing tasks.

Explore how multimodal LLMs extend transformers to handle text and images across modalities. Leverage vision language models to integrate visual and textual information for image captioning and visual question answering.

Leverage generative ai in software testing by using ai chatbots for instant answers and llm-powered testing tools, with examples like Postbot in Postman, Selenium, GitHub Copilot, and Jira plugins.

Compare AI chatbots and LLM-powered testing apps, using GeneAI in both ways, explore their tool integration and automation capabilities, and highlight prompt engineering as essential for effective generative AI testing.

Explore ISTQB official sample exams to practice questions on AI technology types—symbolic AI, classical machine learning, deep learning, and generative AI—plus how context windows limit LLM memory and cross-document referencing.

Analyze tokenization in processing text for large language models, where tokens are building blocks to understand and generate text, and learn how temperature shapes responses.

Explore chapter two of prompt engineering for effective software testing, covering prompt structure, core techniques, and practical tasks like test analysis, design, automated regression, and evaluating AI results.

Explore chapter 2: prompt engineering for effective software testing, covering effective prompt development, applying prompt engineering techniques to testing tasks, and evaluating and refining prompts.

Design precise prompts to elicit accurate test results from LLMs for banking app login testing, specifying positive and negative cases and required fields; avoid vague prompts and hallucinations.

Learn to craft structured prompts for generative AI in software testing by detailing six components: role, context, instruction, input data, constraints, and output format, for precise exam-ready guidance.

Learn about prompt chaining as the first of three main GenAI testing techniques, breaking complex tasks into sequential steps to generate modules and test cases for an e-commerce app.

Master few-shot prompting to guide test case and defect report generation with example-based formats, using one-shot to more-shot examples to balance accuracy, creativity, and consistent CV-style outputs.

Master meta prompting to have AI generate and refine prompts, enabling testers to collaborate with AI systems across prompt chaining and few-shot prompting for robust API security test cases.

Learn how system prompts define large language model behavior, stay static through a session, and differ from user prompts, with examples like Postbot and GitHub Copilot using ISTQB terminology.

Explore how system prompts and user prompts guide LLM responses, driving coordinated results in testing with generative AI, including contrasts between black box and white box testing with examples.

Apply prompt engineering techniques to software testing tasks across analysis, design, automation, and monitoring. Use GenAI to generate test cases, scripts, and insights that reveal risks, gaps, and trends.

Combine prompt chaining, few-shot prompting, and meta-prompting to make prompts more precise, relevant, and usable in real testing. Compare results before and after to measure productivity and defects found.

Explore how generative AI supports test analysis, design, implementation, and execution by generating test conditions, prioritizing tests by risk and business impact, and mapping coverage to requirements.

GeneAI aids test design and implementation by drafting test cases with preconditions, inputs, and expected results, generating synthetic privacy-safe data, and modeling automated scripts for Selenium and Playwright.

Leverage generative AI to automate regression testing, generating scripts from keywords, performing impact analysis and test optimization, and producing actionable reports, dashboards, and enhanced defect insights.

Harness generative AI to monitor tests, analyze unstructured data, and generate insights, dashboards, and reports from test management tools, guiding test control decisions, prioritization and scheduling.

Explore prompt engineering techniques—prompt chaining, few-shot prompting, and meta-prompting—and apply them to test analysis, design, and automation with clear examples like Given-When-Then and defect reports.

Evaluate generative ai results with clear metrics, refine prompts, and review outputs as you would an intern. Compare models and track rejection rates to improve prompt quality and test usefulness.

Explore metrics for evaluating generative AI results on test tasks, including accuracy, precision, recall, relevance and contextual fit, diversity, execution success rate, and time efficiency.

Review and refine prompts through iterative modification based on metrics and results. Use ab testing, output analysis, and stakeholder feedback to optimize prompt length and specificity.

Develop and refine prompts for software test tasks using prompt engineering techniques, evaluate generative AI results, and apply these methods to effective software testing.

Identify ambiguities and inconsistencies in requirements using LLMs. Differentiate AI chatbots from LLM-based testing applications and outline prompt structure concepts such as input data and output format.

Define how system prompts shape LLM behavior and apply prompt chaining to generate, identify potential coverage gaps, and prioritize test conditions and Jerkin-style cases for ISTQB testing with generative AI.

Analyze regression test results using structured prompting to identify discrepancies and cross-check against the known anomalies list. Cluster issues, separate expected and actual results, and highlight discrepancies for regression insights.

Explore using few-shot prompting to generate AI test cases amid the test oracle problem, applying transformation rules and focusing on precision, contextual fit, and diversity metrics.

Explore chapter three: managing risks of generative ai in software testing, covering hallucinations, biases, non-determinism, data privacy and security risks, mitigation techniques, energy impact, and regulation frameworks for exams.

Explore the risks of generative ai in software testing, including hallucinations, reasoning errors, biases, data privacy and security, environmental impact, and ai regulations, standards, and best-practice frameworks.

Explore hallucinations, reasoning errors, and biases in generative ai and their impact on test outputs. Learn mitigation strategies to improve reliability when ai tools generate incorrect or biased results.

Detect and minimize hallucination in llm outputs through cross-verification with requirements or documentation, sme reviews, and consistency checks that align results with system behavior.

Detect reasoning errors in LLM output through logical validation, inspecting the flow, coherence, and structured reasoning, and test procedures with pre-conditions, test cases, and post-conditions, manual or automated.

Detect bias in llm output by reviewing test data to reflect the test strategy and real bank customer scenarios, and note underrepresentation across functional, accessibility, and localization testing.

Mitigate GenAI hallucinations, reasoning errors, and biases in software testing by applying prompt engineering, complete context, and structured prompts; use prompt chaining and model comparisons to verify outputs.

Explore strategies to mitigate non-deterministic behavior of LLMs, including temperature and seed controls, and automated output verification to improve test case stability and reduce hallucinations.

Identify data privacy and security risks in generative ai testing, including unintentional data exposure, data usage controls, and gdpr compliance, and vulnerable test infrastructure, manipulative attacks, and malicious input injection.

Explore data privacy concerns and vulnerabilities in generative ai for test processes and tools, including data exfiltration, request manipulation, data poisoning, and malicious code generation.

Explore GDPR and regulatory context, data minimization, and purpose limitation to protect privacy, while applying secure storage, encryption, access control, and cross-model validation for GenAI testing.

Assess the energy consumption and environmental impact of generative AI in software testing. Limit unnecessary AI interactions and use gen-AI efficiently to reduce CO2 emissions.

Learn AI regulations, standards, and frameworks for gen-AI in software testing, ISO-IEC 42001-2023, ISO-IEC 23053, EU AI Act, and NIST RMF, with transparency, data quality, fault tolerance, and bias mitigation.

Examine risks of generative AI in software testing, including hallucinations, reasoning errors, and biases, along with data privacy and security concerns, energy consumption, and AI regulations and best practice frameworks.

Learn how ISTQB concepts apply to generative AI, including hallucination, bias, and incorrect outputs, plus the impact of clear input data formats on LLM outputs.

Learn to control the LLM outputs by lowering the temperature to narrow the probability distribution, and assess data privacy, data poisoning, and attack vectors in generative AI for software testing.

Address data privacy in generative ai testing with anonymization and encryption, and review energy implications and standards ISO-IEC 23053 and ISO-IEC 42001 for gen-ai testing.

Examine large language model powered test infrastructure, including architectural approaches, components, retrieval, augmented generation, and agents, plus fine tuning and ops to operationalize generative AI.

Explores LLM-powered test infrastructure for software testing, outlines two topics, architectural approaches, fine-tuning your LLM system, and defining LLM Ops for operationalizing generative AI for software testing.

Explore the three components of LLM-powered test infrastructure—front-end, back-end, and LLM engine—and how they analyze requirements, generate tests, and evaluate outputs.

Learn how retrieval augmented generation (RAG) uses embeddings and a vector database to retrieve relevant data and ground LLM outputs with the latest enterprise requirements and test assets.

Explore how LLM-powered agents automate testing tasks through autonomous and semi-autonomous orchestration across Jira and Slack, with human oversight to prevent hallucinations, reasoning errors, and bias.

Discover how fine-tuning LLMs and SLMs on domain data boosts test task accuracy, customizes outputs to company formats, and aligns with internal vocabulary and test processes.

Explore LLM ops for software testing, deploying, and maintaining LLMs across three approaches: AI chatbots, Gen AI test tools, and in-house Gen AI tools, with data privacy and cost considerations.

Explore architectural approaches for LLM-powered test infrastructure, fine-tuning and LLM ops, and operationalizing generative AI for software testing, while previewing chapter five.

Use a retrieval augmented generation (rag) framework to generate banking app test cases by querying one function and retrieving specs from a vector database and tests from a relational database.

Explore a roadmap for deploying and integrating generative AI in test organizations, adoption phases, risks, strategy, LMS selection, building AI capabilities in test teams, and evolving AI enabled test procedures.

Chapter 5 outlines deploying and integrating generative AI in test organizations, offering a roadmap for adoption and change management after implementing prompts, techniques, and LLM ops.

Define test objectives and plan before adopting GenAI in testing, then run pilots to compare models, ensure data quality and compliance, and measure reduction in manual test case writing.

Using unapproved AI tools, shadow AI risks data privacy and information security. Mitigate these risks by addressing regulatory and intellectual property concerns, including GDPR, and potential legal consequences.

Define measurable testing objectives and select compatible LLMs to build a scalable generative ai strategy for software testing, emphasizing data quality, training, measurement, and compliance.

Choose LLMs or SLMs that fit software test tasks by matching capabilities, inputs, reasoning, and outputs with benchmarks, evaluation, licensing, costs, hosting, performance, and fine-tuning potential.

Explore discovery, initiation and usage definition, and utilization and iteration phases to adopt generative AI in software testing, coordinate parallel work streams, address job displacement fears, and monitor scaling.

Implement formal change management to evolve tester roles with generative AI, master prompt engineering, assess LLM capabilities, and enforce privacy and risk mitigation in AI-driven testware.

Develop practical generative AI capabilities in test teams via hands-on training, multi-model prompts, and reusable templates, with learning paths and knowledge sharing to standardize defect reports and testing practices.

The lecture explains how testers evolve from execution specialists to AI-assisted professionals, reviewing AI-generated tests, refining prompts, and maintaining test libraries while leading hybrid teams of humans and Gen-AI agents.

Explore deploying and integrating Gen-AI in test organizations, with a roadmap for adoption in software testing and strategies for managing change when adopting Gen-AI.

Explore shadow AI risks like unauthorized access to sensitive information and define a testing strategy that selects integrable LLMs using task-specific metrics and recurring cost awareness.

Identify the three main phases of adopting generative AI: discovery, initiation, and usage definition; promote a hands-on, incremental learning approach for testers working with large language models.