EU AI act for managers

Learning Objectives

By the end of this chapter, you will be able to:

1. Identify the specific domains where AI is classified as high risk

2. List the obligations that apply to high-risk AI systems

3. Understand what a conformity assessment involves

High-risk AI, the obligations that matter most

High-risk AI systems are where the heaviest compliance burden sits. If your organisation uses or builds AI in any of the domains we'll cover in this chapter, you need to pay close attention. The obligations are real, detailed, and already coming into force.

Which domains are high risk?

Annex III lists the high-risk AI domains

• Biometric identification and categorisation

• Critical infrastructure (energy, water, transport)

• Education: access, assessment, personalised learning

• Employment: recruitment, performance, dismissal decisions

• Essential private and public services: credit, insurance, benefits

• Law enforcement and border control

• Administration of justice

• Democratic processes

The Act lists high-risk AI domains in what's called Annex III. These include biometric identification, critical infrastructure management, education and access to schools, employment and HR decisions, financial services like credit scoring, law enforcement, immigration and border management, administration of justice, and AI that could influence elections. If your AI system operates in any of these areas, even partially, you should treat it as high risk until you have legal confirmation otherwise.

Data governance (Article 10)

High-risk AI must use high-quality data

• Training, validation, and test datasets must be relevant and representative

• Datasets must be free from biases that could create discriminatory outcomes

• Data collection, labelling, and curation must be documented

• Personal data used in training must comply with GDPR

Article 10 sets out requirements for the data used to build and run high-risk AI systems. The data must be relevant to the task, representative of the population the system will affect, and free from biases that could lead to discriminatory outcomes. Everything about how data is collected, labelled, and used must be documented. And if personal data is involved, GDPR requirements apply on top. Data quality is not just a technical concern, it's a legal one.

Technical documentation (Article 11)

You must be able to explain your AI system in detail

• Purpose, capabilities, and limitations of the system

• Design choices and training methodology

• Performance metrics and testing results

• Instructions for use and known risks

Article 11 requires providers of high-risk AI to maintain comprehensive technical documentation. This isn't a brief summary, it's a detailed record covering what the system does, how it was designed and trained, what its performance levels are, and what its known limitations and risks are. This documentation must be maintained throughout the system's lifecycle and be available to regulators on request.

Transparency and instructions for use (Article 13)

Users must be able to understand and use the system correctly

• Systems must be designed to be interpretable by deployers

• Instructions must cover: what the system does, what it doesn't do, how to oversee it

• Limitations and conditions of use must be clear

• Must state when human oversight is required

High-risk AI must come with clear instructions that allow deployers to understand what the system is doing, what it can't do, and how to maintain appropriate oversight. This matters enormously for managers. If you're deploying a high-risk AI tool, you have a responsibility to ensure your team knows how to use it correctly, including when to override it or escalate a decision.

Human oversight (Article 14)

A human must be able to understand, monitor, and intervene

• Systems must allow humans to monitor operation

• Humans must be able to override or halt the system

• Staff using the system must have authority and capability to do so

• Systems must be designed to flag anomalies or unexpected outputs

Article 14 is about keeping humans in control. High-risk AI systems must be designed so that a human can monitor what's happening, understand the outputs, and intervene, including stopping the system, if something goes wrong. This has direct implications for how you train your staff and how you set up your processes. The human overseeing the AI must have genuine capability to act, not just nominal authority.

Conformity assessment

Before deployment, you must assess and document compliance

• Internal assessment against requirements in Articles 8–15

• Some systems require third-party assessment (e.g. biometric ID)

• Result: EU Declaration of Conformity

• System must be registered in the EU AI database

Before a high-risk AI system can be deployed, a conformity assessment must be completed. For most systems, this can be done internally, but it must be thorough and documented. For certain categories, like biometric identification systems, an independent third-party assessment is required. Once conformity is confirmed, the provider issues an EU Declaration of Conformity, and the system is registered in a public EU database.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain what a GPAI model is under the Act

2. Understand the obligations on GPAI providers

3. Explain what "systemic risk" means and why it matters for deployers

General purpose AI, a category that affects almost every organisation

General purpose AI models, things like GPT-4, Claude, Gemini, and similar large language models, are used by millions of organisations. The EU AI Act has a dedicated set of rules for these models, and they're already in force. In this chapter we'll explain what GPAI means under the Act and what it means for your organisation.

What is a GPAI model?

GPAI models are designed to do many different things

• Trained on vast datasets to perform a wide range of tasks

• Not designed for a single specific purpose

• Examples: large language models (LLMs), multimodal models, foundation models

• Can be used directly or integrated into other products and services

A general purpose AI model is exactly what it sounds like, an AI trained to do many things rather than one specific task. Large language models like the ones powering popular AI assistants are the clearest example. They can write, summarise, translate, code, reason, and more. Because they're designed to be flexible and widely deployed, they pose unique regulatory challenges, and the Act treats them accordingly.

Obligations on GPAI providers

Companies that build GPAI models have significant duties

• Must provide technical documentation

• Must publish a summary of training data

• Must implement a copyright compliance policy

• Must cooperate with downstream deployers

If your organisation is building a GPAI model, which for most corporate managers is unlikely but worth noting, you face specific obligations: technical documentation, a summary of training data, a policy to ensure copyright-protected material was used appropriately, and cooperation with businesses that build on top of your model. For most organisations listening to this course, you'll be on the other side of this, as a deployer of GPAI tools.

Systemic risk models

The largest, most powerful GPAI models face additional rules

• Models trained with very large compute (above 10^25 FLOPs) are "systemic risk" models

• Examples: the most powerful frontier AI models

• Additional obligations: adversarial testing, incident reporting, cybersecurity measures

• These designations can change as technology evolves

The Act distinguishes between ordinary GPAI models and those that pose potential systemic risk, the most powerful frontier models trained at massive scale. These face additional requirements including adversarial testing (red-teaming), incident reporting, and cybersecurity obligations. As a deployer of tools built on these models, this affects you indirectly: your vendors must meet these requirements, and you should check that they do.

What this means for deployers

Using GPAI tools comes with responsibilities too

• Assess whether the GPAI tool you use is built on a compliant model

• Understand the terms of use, what is and isn't permitted

• Ensure you're using the tool within its intended scope

• Don't use a GPAI tool for high-risk purposes without a specific conformity assessment

If your organisation uses tools built on GPAI models, and most do, here's what you need to know. First, check whether your vendor is compliant with GPAI obligations. Second, read the terms of use carefully: GPAI models are not licensed for all purposes. Third, do not use a general purpose AI tool for a high-risk application, like making medical or legal decisions, without going through the full high-risk assessment process. A general purpose tool is not automatically cleared for high-risk use.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain the transparency obligations under the Act

2. Distinguish between transparency and explainability

3. Understand what "black box" AI means and when it is acceptable

Transparency and explainability, can you explain what your AI is doing?

One of the most fundamental requirements in the EU AI Act is transparency. People affected by AI systems have a right to know when AI is involved and, in many cases, why a particular decision was made. In this chapter, we'll look at what that means in practice.

Transparency obligations

Users must know when they're interacting with AI

• AI systems interacting with humans must identify themselves as AI

• Deepfake or synthetic content must be labelled as AI-generated

• People subject to AI decisions must be informed

• Information must be clear, accessible, and timely

The basic transparency rule is straightforward: people must know when they're dealing with AI. If your organisation uses a chatbot, users must be told it's a chatbot. If you produce AI-generated content, images, video, text, it must be labelled. If an AI system makes or influences a decision about someone, that person must be informed. These aren't optional courtesies. They're legal requirements.

Explainability

Can you explain why the AI made that decision?

• Explainability = being able to give a meaningful account of how a decision was reached

• Required for high-risk AI decisions affecting individuals

• Not always technically possible with complex models

• The Act requires "interpretability by design" for high-risk systems

Explainability goes a step further than transparency. It's not just about saying "AI was involved", it's about being able to explain why the AI produced a particular output. Why was this candidate ranked lower? Why was this loan application flagged? For high-risk AI systems, the Act requires that the system be designed to be interpretable, so that a meaningful explanation can be given to affected individuals. This is one of the most technically challenging requirements in the Act.

The black box problem

Some AI systems cannot explain themselves, and that's a problem

• Complex neural networks can be opaque even to their creators

• High accuracy does not justify low explainability in high-risk contexts

• Black-box AI is not automatically prohibited, but limits where it can be used

• Explainability tools (SHAP, LIME) can help but have limitations

Many powerful AI systems, particularly large neural networks, are what we call "black boxes." They produce outputs, but even their creators can't fully explain why. This is a genuine tension in AI regulation. High accuracy doesn't justify deploying a system that affects people's lives if you can't explain its decisions. The Act doesn't ban black-box AI outright, but it significantly limits where it can be used. If you're operating in a high-risk domain, you need explainability built in.

Audit trails and accountability

AI decisions must leave a record

• High-risk AI must maintain logs of operation

• Logs must be sufficient to reconstruct decisions after the fact

• Deployers must retain logs for a defined period

• These records are essential for regulatory review and dispute resolution

Linked to transparency is accountability, and accountability requires records. High-risk AI systems must generate and retain logs of their operation. If a decision is challenged, by an employee, a customer, or a regulator, your organisation needs to be able to show what the system did and when. This is not just good practice. It is a legal requirement for high-risk systems.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain what Article 4 requires of organisations

2. Define what "sufficient AI literacy" means in practice

3. Understand how to document and demonstrate compliance

Article 4, Your organisation must ensure staff understand AI

Article 4 of the EU AI Act places a direct obligation on organisations to ensure that staff who work with AI systems have a sufficient level of AI literacy. This is one of the most immediately relevant chapters for managers, because it affects how you hire, train, and support your teams right now.

What Article 4 requires

Providers and deployers must ensure AI literacy among relevant staff

• Applies to both providers and deployers

• Staff involved in operating or overseeing AI must be sufficiently trained

• Literacy must be proportionate to the role and the risk level of the system

• This training programme is part of your compliance journey

Article 4 says that providers and deployers must take measures to ensure their staff have a sufficient level of AI literacy. What does that mean? It means people who work with AI, whether they're building it, deploying it, overseeing it, or making decisions based on its outputs, need to understand what they're working with. The level of literacy required is proportionate to the role and the risk of the system involved.

What "sufficient literacy" looks like

Literacy isn't about becoming an expert, it's about informed use

• Understanding what the AI system does and doesn't do

• Knowing the system's limitations and potential failure modes

• Ability to identify unusual or suspicious outputs

• Understanding when to escalate or override

• Awareness of relevant legal and ethical obligations

Sufficient AI literacy doesn't mean everyone needs to be a data scientist. It means people need to understand the tools they're working with well enough to use them responsibly. Can you tell when the system is producing outputs that seem wrong? Do you know when to escalate? Do you understand the limitations? These are the kinds of questions your staff should be able to answer about any AI system they use. The exact level of knowledge required will vary by role, someone who relies on AI outputs for high-stakes decisions needs deeper literacy than someone using an AI scheduling tool.

Documenting compliance

You must be able to show that training has taken place

• Keep records of training completed by relevant staff

• Document the content of training and its relevance to specific AI systems

• Update training when new AI tools are deployed or existing ones change

• Consider role-based training plans for different levels of AI interaction

It's not enough to run training, you need to be able to demonstrate that you did. That means records: who was trained, on what, when, and how that training relates to the AI systems they work with. If a regulator asks for evidence of AI literacy compliance, your organisation needs to be able to produce it. This training programme you're participating in right now is part of that evidence trail.

Learning Objectives

By the end of this chapter, you will be able to:

1. Distinguish between human-in-the-loop and human-on-the-loop oversight models

2. Explain when and how to override an AI decision

3. Describe an effective escalation path for questionable AI outputs

Keeping humans in control, oversight and escalation

The EU AI Act is built on a fundamental principle: AI should support human decision-making, not replace it, especially in high-stakes situations. In this chapter, we'll look at how to structure human oversight in practice, and what to do when something goes wrong.

Types of human oversight

Three models of human oversight

• Human-in-the-loop: a human approves each AI output before action is taken

• Human-on-the-loop: AI acts autonomously but a human monitors and can intervene

• Human-in-command: a human can halt or modify the AI system at any time

There are different ways to maintain human oversight of AI systems. In a human-in-the-loop model, a human approves every decision before it takes effect, the AI recommends, the human decides. In a human-on-the-loop model, the AI acts automatically but a human monitors in real time and can intervene. Human-in-command is a broader concept referring to the ability to modify or shut down the system at any time. For high-risk AI systems, the Act effectively requires at minimum a human-on-the-loop approach, with meaningful ability to intervene.

When to override an AI decision

Human override is not a failure, it is a feature

• Override when the output seems factually wrong or inconsistent

• Override when the decision would affect someone unfairly

• Override when the system is operating outside its intended context

• Override when you don't understand why the output was produced

• Document every override and the reason for it

One of the cultural challenges with AI is the tendency to trust it, especially when it's confident-sounding. But high confidence does not mean correct. Your staff must feel empowered to override AI outputs when something doesn't seem right. When should they? When the output is factually wrong. When it would treat someone unfairly. When the system seems to be operating in a context it wasn't designed for. When the reasoning is opaque. Every override should be documented, both for compliance and for improving the system over time.

Escalation paths

Know who to call when AI produces a questionable output

• Define a clear escalation path before deployment

• Frontline staff → Team manager → AI/compliance lead → Legal

• Escalate when: output is unexplained, affects a protected characteristic, or involves a significant decision

• Document escalations as part of your AI governance record

Every AI deployment in a high-risk area should come with a defined escalation path. Frontline staff need to know what to do when something seems wrong, and who to contact. That might be a team manager, an AI lead, a compliance officer, or a legal team, depending on the severity. The key is that the path is clear before a problem arises, not decided in the moment. Escalations should be logged as part of your governance record.

Building a culture of oversight

Human oversight only works if people feel empowered to use it

• Staff must know they are authorised to question AI outputs

• No blame culture for overrides, they are part of the system

• Managers should model the behaviour: question AI, don't just accept it

• Regular review of override and escalation data improves the system

Formal oversight mechanisms are only as good as the culture they sit in. If staff feel that questioning or overriding AI is risky, that it will be seen as slowing things down or second-guessing management, they won't do it. As a manager, your job is to make it clear that oversight is expected, valued, and safe. Model the behaviour yourself: question AI outputs, take time to verify, reward the people who catch errors.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain how the EU AI Act and GDPR interact

2. Apply data minimisation and purpose limitation principles to AI use

3. Identify the risks of using personal or sensitive data in AI systems

AI and data protection, two frameworks, one responsibility

The EU AI Act doesn't replace GDPR, it sits alongside it. If your AI system processes personal data, both frameworks apply simultaneously. In this chapter we'll look at where they overlap, where they diverge, and what that means for your organisation in practice.

How GDPR and the AI Act interact

Both laws apply at the same time, and reinforce each other

• GDPR governs the lawful use of personal data

• The AI Act governs the use of AI systems that process that data

• Non-compliance with GDPR can also mean non-compliance with the AI Act

• Data protection impact assessments (DPIAs) and AI risk assessments often overlap

Think of GDPR and the AI Act as two lenses on the same situation. GDPR asks: are you handling personal data lawfully, fairly, and securely? The AI Act asks: is your AI system appropriate, transparent, and safe? If your AI system processes personal data, and most do, you need to satisfy both. A GDPR Data Protection Impact Assessment and an AI Act risk assessment will often cover very similar ground, and running them together is more efficient.

Lawful basis for using data in AI

You need a valid legal reason to use personal data in AI

• Consent: freely given, specific, informed, unambiguous

• Legitimate interests: must not override individuals' rights

• Contract: data use is necessary for the contract with the individual

• Legal obligation: you're required to process the data by law

• AI systems often rely on legitimate interests, but this must be documented

Under GDPR, you need a lawful basis to process personal data. This applies just as much when that data is used to train or run an AI system. Consent is the most robust basis but also the hardest to maintain at scale. Legitimate interests is commonly used, but it requires a balancing test, your interests must not override individuals' fundamental rights. Whatever basis you use, it must be documented and reviewable.

Data minimisation and purpose limitation

Collect only what you need, use it only for its intended purpose

• Data minimisation: collect only the data necessary for the specific AI task

• Purpose limitation: don't use data collected for one purpose to train AI for another

• Re-using historical data in AI systems is a common compliance risk

• Document the specific purpose of every dataset used in AI development

Two GDPR principles are especially important in AI contexts. Data minimisation means you should only collect and use the data you genuinely need for the AI task. Purpose limitation means you can't take data collected for one reason, say, customer service records, and use it to train an AI system for a completely different purpose without fresh consent or a new legal basis. This catches many organisations out. Just because you have data doesn't mean you can use it to train AI.

Special category and sensitive data

Some data types require extra protection in AI systems

• Special categories: health, biometric, racial/ethnic origin, political opinion, religion

• Processing requires explicit consent or specific legal exemptions

• AI systems trained on or producing sensitive data face heightened scrutiny

• Bias and discrimination risks are highest with sensitive data

GDPR's special category data, health records, biometric data, information about ethnicity, political views, religious beliefs, requires explicit consent or very specific legal exceptions to use. AI systems that train on or produce this kind of data face the highest scrutiny. They're also where bias and discrimination risks are most acute. If your AI system touches any of these data types, you should seek specific legal advice before proceeding.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain how bias enters AI systems

2. Describe the types of discriminatory outcomes AI can produce

3. List strategies for identifying and mitigating bias

Bias in AI, where it comes from and what to do about it

AI systems don't just inherit the biases of the humans who build them, they can amplify them at scale. In this chapter, we'll look at how bias enters AI systems, what discriminatory outcomes look like in practice, and what your organisation can do to identify and address them.

How bias enters AI systems

Bias is baked in long before the AI is deployed

• Historical bias: training data reflects past inequalities

• Representation bias: some groups are underrepresented in training data

• Measurement bias: proxies for outcomes introduce distortion

• Feedback loops: biased outputs reinforce biased inputs over time

Bias doesn't appear from nowhere. It gets into AI systems through the data they're trained on. If your historical hiring data shows that men were promoted more than women, an AI trained on that data will learn to favour men. If your training dataset underrepresents certain ethnic groups, the system will perform worse for those groups. And if a biased system's outputs become future training data, the bias compounds over time. This is not a hypothetical. It has happened, in hiring tools, in facial recognition, in healthcare diagnostics.

Types of discriminatory AI outcomes

Discrimination can be direct or indirect

• Direct discrimination: explicitly using a protected characteristic (race, gender, age)

• Indirect discrimination: using a proxy variable that correlates with a protected characteristic

• Example: using postcode as a variable that correlates with ethnicity

• Disparate impact: the system produces systematically worse outcomes for a protected group

Discrimination in AI can be obvious or hidden. Direct discrimination means the AI is explicitly using a protected characteristic like race or gender to make decisions, clearly illegal. But indirect discrimination is more common and harder to spot. Using postcode as a variable in a credit model seems neutral, but if postcodes correlate with ethnicity, the effect is discriminatory. What matters legally is not intent, but outcome. If the system produces systematically worse results for a protected group, that is a problem regardless of whether it was intentional.

Fairness audits

You need to test for bias proactively

• Disaggregate outputs by protected characteristics

• Compare outcomes across groups, do some groups consistently do worse?

• Test with representative datasets, including edge cases

• Use independent auditors for high-risk systems

• Document all testing and results

The only way to know if your AI system is discriminating is to test it. That means breaking down outputs by protected characteristics, gender, age, ethnicity, disability, and asking whether some groups consistently receive worse outcomes. This needs to be done with representative data, including people at the margins. For high-risk AI systems, independent auditing is strongly recommended. And all testing and results must be documented.

Mitigation strategies

Bias can be reduced, but not always eliminated

• Diverse development teams produce more equitable AI

• Careful data curation and bias testing at training stage

• Post-deployment monitoring and feedback loops

• Human review for decisions affecting protected groups

• Transparency in how variables are used

Mitigation starts at the design stage. Diverse teams are less likely to produce systems with blind spots. Careful data curation, removing or rebalancing biased training data, reduces the problem at source. Post-deployment monitoring catches drift and emerging bias. And maintaining human review for decisions that affect people in protected groups provides a crucial safety net. There is no silver bullet, bias mitigation is ongoing work, not a one-time fix.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain the penalties for non-compliance with the EU AI Act

2. Describe how civil liability may arise from AI-related harms

3. Identify practical steps to reduce legal exposure

What happens if things go wrong, liability and sanctions

The EU AI Act has teeth. Non-compliance can result in significant fines, civil liability, and reputational damage. In this chapter, we'll look at what the sanctions are, how liability works in practice, and what your organisation can do to stay on the right side of the law.

Administrative fines

Fines are substantial, and proportionate to the breach

• Violating prohibited practices (Article 5): up to €35 million or 7% of global annual turnover

• Other violations of the Act: up to €15 million or 3% of global annual turnover

• Providing false information to authorities: up to €7.5 million or 1% of turnover

• Higher of the two figures applies; SMEs may receive proportionate penalties

The fines in the EU AI Act are comparable to GDPR penalties. Violating the prohibited practices, the outright bans we discussed in Chapter 4, can result in fines of up to 35 million euros or 7% of global annual turnover, whichever is higher. Other breaches can attract fines up to 15 million or 3% of turnover. These are not theoretical numbers. Regulators are being resourced to enforce this law, and early enforcement cases will set strong signals.

Civil liability

Individuals harmed by AI may also bring civil claims

• The EU AI Liability Directive (companion legislation) eases burden of proof for claimants

• Organisations may need to disclose AI system logs in court proceedings

• Liability can fall on providers, deployers, or both

• Insurance and indemnity clauses in vendor contracts become critical

Beyond regulatory fines, there is civil liability to consider. The EU has also been developing an AI Liability Directive that makes it easier for individuals harmed by AI to bring claims, including by requiring organisations to disclose their AI system logs in proceedings. This means organisations must maintain good records, not just for regulatory compliance, but for their own legal defence. And it makes vendor contracts, specifically who is responsible for what, extremely important.

How to reduce legal exposure

Compliance is your best defence

• Conduct and document risk assessments before deployment

• Maintain technical documentation and audit logs

• Train staff and keep records of training

• Have clear escalation and override procedures

• Review vendor contracts for liability allocation

• Establish an internal AI governance function

The best way to manage legal risk is to do the compliance work properly in the first place. That means conducting and documenting risk assessments, maintaining technical records, training your staff, keeping logs, and having clear procedures for oversight and escalation. It also means making sure your contracts with AI vendors clearly allocate responsibility, you don't want to find out in a crisis that the liability for a vendor's AI failure has defaulted to you.

Learning Objectives

By the end of this chapter, you will be able to:

1. Describe the key elements of an AI governance structure

2. Draft the components of an internal AI usage policy

3. Explain what change control means in an AI context

Building your AI governance framework

Compliance with the EU AI Act isn't a one-time project, it requires an ongoing governance structure. In this chapter, we'll look at how to set up the internal controls, policies, and processes that will keep your organisation compliant as AI use grows and evolves.

The AI governance structure

Someone needs to be in charge of AI across your organisation

• Designate an AI lead or AI Compliance Officer

• Establish an AI review committee or risk board

• Define escalation paths from business units to governance function

• Ensure board-level visibility of AI risks and compliance status

Good AI governance starts with clear ownership. Someone in your organisation needs to be responsible for AI compliance, whether that's a dedicated AI Compliance Officer, an extension of your existing compliance function, or a shared responsibility between legal, technology, and risk teams. An AI review committee, with representatives from legal, technology, HR, and business units, provides collective oversight. And AI risk should be visible at board level, not buried in a technical team.

Internal AI usage policy

Your policy should answer: what can we use, and how?

• Approved AI tools: which tools are permitted for which purposes

• Prohibited uses: what your organisation will never do with AI

• Approval process: how staff request new AI tools

• Data handling rules: what data can be entered into AI systems

• Disclosure requirements: when and how to tell users AI is involved

Every organisation should have an internal AI usage policy. This doesn't need to be a complex legal document, it needs to be clear and practical. It should tell employees which AI tools are approved and for what purposes. It should clearly state what is prohibited. It should explain how to request a new AI tool. And it should set out the rules around what data can be entered into external AI systems, a critical point given that many employees are using public AI tools and may unknowingly input sensitive company or customer data.

Change control and versioning

When AI systems change, your compliance records must keep up

• AI models are updated by vendors, often without notice

• Significant updates may change the risk profile of a system

• Establish a process for reviewing AI tools when major updates occur

• Maintain version history of any AI tools your organisation has assessed

One of the trickier aspects of AI governance is that the systems themselves change. A vendor might update their model, sometimes without prominent notification, in ways that affect the system's behaviour and potentially its risk classification. Your governance process needs to include a mechanism for tracking updates and triggering a re-assessment when significant changes occur. This is especially important for high-risk AI systems.

Approval workflows

New AI tools should go through a review process before deployment

• Intake: who submits a request and what information is required

• Assessment: risk classification, GDPR review, vendor assessment

• Approval: who has authority to approve at each risk level

• Deployment: training, documentation, monitoring plan

• Review: scheduled reassessment after deployment

Treat AI tool adoption like any other significant IT or procurement decision, with a structured approval workflow. A new AI tool should go through intake, risk classification, data protection review, and vendor assessment before being deployed. Approval authority should match the risk level, low-risk tools might be approved by a team manager, while high-risk tools require sign-off from the governance committee and legal team. And every deployment should come with a monitoring plan and a scheduled date for review.

Learning Objectives

By the end of this chapter, you will be able to:

1. Explain what due diligence is required when procuring AI tools

2. Identify the contractual protections your organisation needs

3. Understand how vendor compliance affects your own compliance

Buying AI, the compliance questions you must ask before you sign

Most organisations don't build their AI systems from scratch, they buy them. But buying AI doesn't absolve you of responsibility. As a deployer, your compliance depends in part on your vendor's compliance. In this chapter, we'll look at what to ask, what to check, and what to put in your contracts.

Due diligence before procurement

Assess compliance before you buy

• What risk category does this system fall into?

• Has a conformity assessment been completed?

• Is the system registered in the EU AI database (if required)?

• What is the vendor's data processing and security posture?

• Does the vendor have documentation and audit trails available?

Before signing any contract for an AI system, you should be asking your vendor a set of standard questions. What risk category does the system fall into under the EU AI Act? If it's high risk, has a conformity assessment been completed and is there documentation available? Is the system registered in the EU AI database? What personal data will the system process, and how is it protected? These aren't optional questions, they're the basis of responsible procurement.

Contractual protections

Your contract should allocate responsibility clearly

• Provider obligations: technical documentation, conformity assessment, incident notification

• Your right to audit and inspect documentation

• Data processing agreement (DPA) aligned with GDPR

• Indemnity clauses: who bears liability if the system causes harm

• Notification obligations: vendor must inform you of significant updates or incidents

Your contract with an AI vendor needs to clearly set out who is responsible for what. The vendor, as provider, should commit to maintaining technical documentation and completing conformity assessments. You should have the right to access that documentation and to audit compliance. A data processing agreement must be in place if personal data is involved. And crucially, there should be indemnity provisions, agreed in advance, for what happens if the system causes harm. Vendors who refuse these terms should be treated with caution.

Ongoing vendor monitoring

The relationship doesn't end at contract signature

• Schedule regular reviews of vendor compliance status

• Require notification of model updates or significant changes

• Monitor for incidents or regulatory actions involving your vendor

• Include exit provisions in contracts in case of vendor non-compliance

Procurement is the beginning of the relationship, not the end. You should have regular checkpoints with significant AI vendors to review their compliance status. Contracts should require notification if the model is updated in ways that affect its behaviour. You should monitor whether your vendor faces any regulatory action, their compliance problems can become yours. And your contracts should include exit provisions so that if a vendor becomes non-compliant, you can transition away without being locked in.

Learning Objectives

By the end of this chapter, you will be able to:

1. Define what constitutes a "serious incident" under the Act

2. Explain the incident reporting process and timelines

3. Describe the post-market monitoring obligations for high-risk AI

When things go wrong, incident reporting and ongoing monitoring

The EU AI Act doesn't just regulate AI at the point of deployment. It imposes ongoing obligations, including the requirement to monitor high-risk AI systems after they go live, and to report serious incidents to national authorities. In this chapter, we'll look at what that means in practice.

What is a serious incident?

Not every error is a reportable incident, but some are

• Death or serious harm to a person caused or contributed to by a high-risk AI system

• Serious breach of fundamental rights

• Serious damage to property or the environment

• Unexpected and significant deviation from intended performance

A serious incident, under the Act, is not just any malfunction or error. It's a specific category: where a high-risk AI system has caused or contributed to death, serious physical injury, a significant breach of someone's fundamental rights, or major property damage. It also covers significant unexpected deviations from the system's intended performance, even if harm hasn't occurred yet. Think of it like a near-miss reporting system.

Reporting obligations

Serious incidents must be reported promptly to national authorities

• Providers must report to the relevant national market surveillance authority

• Deployers must notify providers when they identify a serious incident

• Timelines: immediately for life-threatening incidents; 15 days otherwise

• Reports must include: description of incident, system details, actions taken

If a serious incident occurs, it must be reported. Providers report directly to the national market surveillance authority in the relevant EU member state. Deployers, that's most organisations using bought-in AI, have a duty to notify their vendor (the provider) promptly when they identify a serious incident. The timelines are tight: life-threatening situations require immediate notification; other serious incidents within 15 days. You need processes in place to identify, escalate, and report incidents before they happen.

Post-market monitoring

You must actively monitor high-risk AI after deployment

• Providers must have a post-market monitoring plan

• Deployers must monitor for unexpected outcomes and report to providers

• Monitoring includes: performance metrics, bias indicators, user feedback

• Schedule regular reviews, at minimum annually for high-risk systems

Deploying a high-risk AI system is not the end of your obligations, it's the beginning of an ongoing monitoring responsibility. You must actively track how the system is performing: is it producing the expected results? Are there signs of drift, bias, or unexpected failure modes? Are users flagging problems? This monitoring should be formal, documented, and scheduled. For high-risk systems, a review at least annually is the baseline, more frequently if the system operates in fast-changing conditions.

Learning Objectives

By the end of this chapter, you will be able to:

1. Conduct a basic AI inventory for your organisation

2. Identify compliance gaps against EU AI Act requirements

3. Build a prioritised compliance roadmap

Where are you now, and how do you get to compliance?

Understanding the law is the first step. Applying it to your organisation is the second. In this chapter, we'll walk through how to conduct a gap analysis, mapping where you are today against where the law requires you to be, and how to turn that into a realistic, prioritised compliance roadmap.

Step 1: AI inventory

Start by knowing what AI you actually use

• Survey all business units for AI tools in use (including informal/shadow AI)

• Include: commercial tools, internal builds, AI embedded in other software

• Capture: tool name, purpose, data processed, who uses it, vendor details

• Many organisations are surprised by how many AI tools they find

You can't manage what you don't know about. The first step is to build a complete inventory of AI tools in use across your organisation. This means going beyond the IT-approved list, many teams are using AI tools informally, without central approval. Survey every business unit. Ask specifically about AI features embedded in tools they already use, Microsoft 365, Salesforce, HR systems, because these often go unnoticed. The inventory will almost certainly be longer than you expect.

Step 2: Risk classify each system

Once you know what you have, classify each tool

• Apply the risk tier framework from Chapter 3 to each tool

• High-risk classifications trigger the full compliance checklist

• Document your classification rationale for each system

• When in doubt, classify upward and seek legal advice

Once you have your inventory, apply the risk classification framework from Chapter 3 to each system. Be honest and conservative, if there's genuine doubt about whether a system is high risk, treat it as high risk until you have legal confirmation otherwise. Document your reasoning for each classification. This documentation protects you if your rationale is ever questioned by a regulator.

Step 3: Gap assessment

For each system, what's in place and what's missing?

• For high-risk systems: is there technical documentation? Conformity assessment? Human oversight? Logging?

• For all systems: are transparency obligations met? Are staff trained?

• For data: do GDPR and data minimisation requirements apply and are they met?

• For governance: is there a policy, a governance structure, vendor contracts?

For each AI system in your inventory, work through the applicable obligations and assess what's currently in place. For high-risk systems, that means checking documentation, conformity assessments, oversight mechanisms, and logs. For all systems, check transparency and staff training. Across the board, review whether GDPR requirements are met and whether your governance infrastructure is adequate. This produces your gap list, the specific things your organisation needs to fix or build.

Step 4: Build the roadmap

Prioritise by risk, then by implementation effort

• Priority 1: gaps relating to prohibited practices, fix immediately

• Priority 2: gaps in high-risk systems with live deployment

• Priority 3: governance and policy infrastructure

• Priority 4: training and documentation

• Set realistic timelines and assign clear ownership for each action

Turn your gap list into a roadmap. Prioritise ruthlessly. Anything relating to a prohibited practice, something the Act bans, needs to be addressed immediately. High-risk systems already in live deployment are the next priority. Then work on building your governance infrastructure and getting policies in place. Training and documentation can often be progressed in parallel. Each action needs a clear owner and a realistic deadline. A roadmap that isn't owned by specific people is just a wish list.

Learning Objectives

By the end of this chapter, you will be able to:

1. Identify sector-specific AI regulations that overlap with the EU AI Act

2. Explain the key AI compliance considerations in financial services, healthcare, and HR

3. Navigate the interaction between the AI Act and sector regulation

The AI Act doesn't operate in isolation, sector rules apply too

The EU AI Act is a horizontal regulation, it applies across all sectors. But many sectors have their own specific AI-related rules and guidelines on top of the Act. In this chapter, we'll look at three sectors where this overlap is particularly significant: financial services, healthcare, and HR. If your organisation operates in one of these areas, you'll need to comply with both layers.

Financial services

AI in financial services faces the AI Act AND sector-specific rules

• Credit scoring AI: high risk under Annex III; also subject to consumer credit directives

• Algorithmic trading: financial market regulations apply alongside AI rules

• Anti-money laundering AI: subject to AML directives as well as the AI Act

• MiCA (crypto regulation) intersects with AI used in crypto markets

• EBA and ESMA guidance on AI use in supervised entities

In financial services, AI used for credit scoring, insurance underwriting, or fraud detection is classified as high risk under the AI Act, but it's also subject to existing financial regulation. The European Banking Authority and the European Securities and Markets Authority have issued guidance on AI use in supervised firms. Anti-money laundering AI must also comply with EU AML directives. The practical implication is that your compliance programme needs to satisfy both the AI Act and your sectoral regulator, and their requirements often reinforce each other.

Healthcare

AI in healthcare is high risk, and also regulated as a medical device

• Diagnostic AI, clinical decision support, patient triage tools: high risk under Annex III

• Medical device AI may also fall under the EU Medical Devices Regulation (MDR)

• Dual regulation: both conformity assessment regimes may apply

• Patient data: GDPR special category data protections apply throughout

• Clinical validation requirements may exceed what the AI Act requires

Healthcare AI sits at the intersection of the AI Act and the EU Medical Devices Regulation. An AI system used to support clinical diagnoses or treatment decisions is likely high risk under both frameworks. Clinical validation requirements in the MDR are often more demanding than the AI Act alone. And because healthcare AI processes health data, a special category under GDPR, data protection obligations are heightened. If your organisation operates in healthcare, you almost certainly need specialist legal and regulatory advice.

Human resources

AI in HR is high risk, and touches employment law across member states

• Recruitment and selection AI: explicitly listed as high risk in Annex III

• Performance monitoring, promotion decisions, dismissal support tools: also high risk

• Employee data: special sensitivities around consent and the employment relationship

• National employment law varies: what's lawful in one member state may not be in another

• Works council and trade union consultation requirements in some jurisdictions

HR is one of the most common areas where organisations are already using AI, and one of the areas where the compliance burden is highest. Recruitment tools, performance management systems, and anything that supports dismissal decisions are all explicitly classified as high risk. On top of the AI Act, you need to consider national employment law, which varies significantly across EU member states. In some countries, introducing AI in the workplace requires consultation with employee representative bodies. Your HR and legal teams need to work together on this.

Learning Objectives

By the end of this chapter, you will be able to:

1. Understand your specific obligations based on your role

2. Apply AI Act requirements to your day-to-day responsibilities

3. Know who to work with across the organisation to ensure compliance

Your role in AI compliance, what it means for you specifically

Everyone in an organisation has a role to play in AI compliance, but the specific responsibilities differ depending on what you do. In this chapter we'll break down what the EU AI Act means for three key groups: developers and data scientists, compliance and legal teams, and business managers. Find your section and focus there.

For developers and data scientists

Technical teams carry significant provider-level obligations

• Document model design, training data, and testing methodology

• Conduct and record bias testing before deployment

• Build explainability and logging into systems from the start

• Flag risk classification questions to legal, don't assume minimal risk

• Maintain technical documentation throughout the model lifecycle

If you work in a technical role, building, training, or maintaining AI systems, you carry some of the heaviest obligations under the Act. Documentation is your core responsibility: design choices, data sources, testing results, known limitations. Explainability and logging need to be built into systems from the start, not retrofitted. And you need to work closely with your legal and compliance colleagues on risk classification, don't assume a system is minimal risk without checking.

For compliance and legal teams

Compliance teams are the nerve centre of AI governance

• Own the AI inventory and risk classification process

• Lead conformity assessments for high-risk systems

• Draft and maintain AI usage policies and vendor contract templates

• Act as the liaison with national market surveillance authorities

• Monitor regulatory developments and update internal guidance

Compliance and legal teams are the organisational hub for AI Act compliance. Your responsibilities include owning the AI inventory, leading risk classification and conformity assessments, drafting policies and contract templates, and being the primary interface with regulators. You also need to stay current with regulatory developments, guidance, enforcement decisions, and changes to the Act itself, and translate those into updated internal guidance for the rest of the organisation.

For business managers

Managers are the frontline of human oversight

• Know the risk classification of AI tools your team uses

• Ensure your team is trained and knows when to escalate

• Review AI outputs critically, don't rubber-stamp

• Raise procurement requests through the approved governance process

• Report anomalies and incidents to the appropriate team

As a business manager, you are the most important line of defence in making AI work safely and compliantly. You don't need to be a technical expert, but you do need to know the risk classification of the tools your team uses, ensure your people are trained, and create a culture where questioning AI outputs is normal and expected. When something seems wrong, escalate it. When your team wants to use a new AI tool, use the approved process. Your oversight is not a formality, it's a legal requirement.

Working across functions

AI compliance requires everyone to work together

• Technology and legal must work together on risk classification and documentation

• HR and legal must collaborate on employee-facing AI tools

• Procurement and compliance must co-design vendor assessment processes

• Senior leadership must provide visible sponsorship of compliance

No single team can deliver AI Act compliance alone. Technology teams need legal oversight. Legal teams need technical knowledge. HR, finance, and operations all have AI in their function that needs to be assessed. And none of it works without genuine commitment from senior leadership. AI compliance is a cross-functional programme, and it needs to be treated as one.

Learning Objectives

By the end of this chapter, you will be able to:

1. Apply the risk classification framework to real AI deployments

2. Identify what made historical AI failures non-compliant

3. Draw lessons from successful AI deployments

Learning from real deployments, what good and bad look like

Theory is important, but real cases bring the rules to life. In this chapter, we'll work through a set of industry case studies: some where AI was deployed successfully and compliantly, and others where things went badly wrong. We'll apply what we've learned to understand why.

Case study 1: Amazon's recruitment AI (failure)

A high-profile AI hiring failure

• Amazon built an AI to screen CVs, abandoned in 2018

• System trained on 10 years of historical hiring data (predominantly male)

• Learned to penalise CVs containing the word "women's" (e.g. women's chess club)

• Outcome: discriminatory, biased against female candidates

• Under the AI Act: high-risk system, likely non-compliant on bias and documentation grounds

Amazon's AI recruitment tool became one of the most cited examples of AI bias. Trained on a decade of historical hiring data from a predominantly male workforce, the system learned patterns that disadvantaged women, penalising CVs that included terms associated with female applicants. Amazon scrapped it before it went live. Under the EU AI Act, this system would be classified as high risk, recruitment is explicitly listed in Annex III. A thorough bias testing programme, as required by Article 10, should have caught this before deployment.

Case study 2: AI in healthcare diagnostics (success)

AI supporting, not replacing, clinical judgment

• Several NHS trusts use AI to detect early signs of diabetic retinopathy from retinal scans

• AI flags scans for clinical review, a human always makes the final decision

• Validated against large diverse datasets; performance monitored continuously

• Transparent to patients that AI is involved in the screening process

Contrast that with AI used in diabetic eye screening in the NHS. The system analyses retinal scans and flags those showing signs of diabetic retinopathy for clinical review. Crucially, a human clinician always makes the final call, the AI supports, not replaces, clinical judgment. The system was validated on diverse datasets, performance is continuously monitored, and patients are informed. This is close to what good AI deployment looks like: high-risk system, properly assessed, with robust human oversight and transparency.

Case study 3: Facial recognition in retail (failure)

Mass biometric surveillance without consent

• Several UK retailers trialled facial recognition to identify "known shoplifters"

• No meaningful consent from members of the public

• ICO found the processing unlawful under GDPR

• Under the EU AI Act: biometric identification is high risk; real-time surveillance in public spaces may be prohibited

• Reputational damage significant

Several UK retailers experimented with facial recognition in their stores to identify people previously associated with shoplifting. The Information Commissioner's Office found this unlawful, people were being identified without consent in a private space they had no meaningful choice but to enter. Under the EU AI Act, biometric identification is high risk, and certain forms of real-time biometric identification may fall into the prohibited category. Beyond the legal exposure, the reputational damage to the brands involved was significant and lasting.

Case study 4: AI for content moderation (nuanced)

A case where the right answers aren't black and white

• Large platforms use AI to detect hate speech, misinformation, and harmful content

• High volume means AI is the only feasible approach

• Risk of false positives: legitimate speech incorrectly removed

• Risk of false negatives: harmful content missed

• Under the AI Act: transparency obligations; human review processes required

Content moderation is a case where AI is genuinely necessary at scale, but where the risks run in both directions. False positives silence legitimate speech. False negatives allow harmful content to spread. Under the AI Act, platforms using AI content moderation must be transparent about it and must have human review processes for contested decisions. This case illustrates that "risky" doesn't always mean "avoidable", sometimes it means "requires especially careful governance."

Learning Objectives

By the end of this chapter, you will be able to:

1. Assess the compliance implications of common AI tools your organisation uses

2. Apply the Act's requirements to generative AI, chatbots, and recommender systems

3. Identify the most common compliance gaps with these tools

The tools you're probably already using, and what the Act says about them

Most organisations have already adopted AI tools, often faster than their governance has kept up. In this chapter, we'll look at three of the most common: generative AI tools, chatbots, and recommender systems. For each, we'll apply what we've learned and identify the compliance considerations that matter most.

Generative AI tools (e.g. AI assistants and writing tools)

GenAI in the workplace is generally limited or minimal risk, with important caveats

• Typically limited or minimal risk as a general productivity tool

• Key risk: what data employees enter into these tools

• AI-generated content must be disclosed where required (public communications, official documents)

• Not appropriate for high-risk decisions without additional assessment

• Employees need clear guidance on acceptable and prohibited uses

Generative AI assistants used for writing, summarising, and brainstorming are generally in the limited or minimal risk category, the tools themselves are not inherently high risk. But the risks come from how they're used. Employees entering confidential customer data, personal information, or sensitive business data into public AI tools is a major compliance risk, both under GDPR and under company policy. And using a general-purpose AI tool to support high-stakes decisions, medical, legal, financial, without a proper assessment is not acceptable. Clear employee guidance is essential.

Customer service chatbots

Chatbots are limited risk, transparency is the main obligation

• Must disclose to users that they are interacting with AI

• The disclosure must be clear and at the start of the interaction

• Must be able to escalate to a human if requested

• If chatbot collects personal data: full GDPR obligations apply

• If chatbot makes or influences consequential decisions: may be high risk

Customer-facing chatbots are typically classified as limited risk, the main obligation is transparency. Users must be told, clearly and at the outset, that they are interacting with an AI system. The ability to request a human must be available. If the chatbot collects personal data, which most do, GDPR applies fully. And if the chatbot influences consequential decisions, credit, benefits eligibility, healthcare triage, the risk level increases significantly and may tip into high risk.

Recommender systems

Recommenders range from minimal to high risk depending on context

• Product recommendations for consumers: typically minimal risk

• Content recommendation on news or social platforms: limited risk (transparency obligations)

• Recommendations influencing access to services: potentially high risk

• Watch for: filter bubbles, reinforcing bias, manipulation through personalisation

Recommender systems, the algorithms that decide what you see, read, or are offered, span the whole risk spectrum. A product recommender in an e-commerce context is typically minimal risk. A content recommender on a news platform is limited risk, with transparency obligations around algorithmic curation. A system that recommends which customers should be offered a loan, or which job applicants should progress, is high risk. Whatever the context, recommenders deserve scrutiny for bias, filter bubble effects, and whether their personalisation crosses into manipulation.

Learning Objectives

By the end of this chapter, you will be able to:

1. Identify AI misuse and compliance failures in realistic scenarios

2. Apply ethical reasoning frameworks to AI dilemmas

3. Make and defend a decision in ambiguous AI situations

What would you do? Ethical dilemmas and red flags in AI

This chapter is interactive. We're going to work through a series of real-world scenarios. In each case, pause the video, think about what you would do, then listen to the analysis. There are no trick questions, but many of these cases are genuinely ambiguous, and that's the point. Good AI judgment isn't about rules alone. It's about reasoning.

Scenario 1: The efficient shortcut

Your team wants to use AI to automate a benefits eligibility decision

Scenario: Your operations team has found an AI tool that can process benefits eligibility assessments ten times faster than the current manual process. The vendor says it's been used by other organisations. No risk assessment has been done. Pressure is high to deploy quickly.

The question: What do you do?

Analysis: Benefits eligibility is explicitly listed as a high-risk domain in Annex III. No risk assessment, no conformity assessment, no documentation, this system cannot be deployed in this state. Speed pressure is not a justification for non-compliance. The right answer is to initiate the proper assessment process, understand the vendor's compliance status, and only deploy when requirements are met. Pushing back on speed pressure is part of your job as a manager.

Scenario 2: The hallucinating AI

Your colleague is submitting AI-generated research as their own work

Scenario: You notice a colleague regularly using an AI tool to generate research summaries that they submit as their own analysis. One of these summaries contains a factual error, the AI has confidently stated something that isn't true. The colleague hasn't checked it.

The question: What are the issues, and what do you do?

Analysis: There are two separate issues here. First, AI-generated content presented as personal analysis may be a question of professional integrity or company policy, worth raising with the colleague directly or with management. Second, and more urgently, unchecked AI hallucinations entering decision-making processes is a genuine risk, especially if the output is used to inform consequential decisions. This is exactly why human oversight and verification are required. Raise the specific factual error immediately. Use it as an opportunity to discuss AI verification practices with your team.

Scenario 3: The vendor promise

A vendor tells you their AI hiring tool is "fully GDPR compliant"

Scenario: You're evaluating an AI recruitment tool. The vendor assures you it is "fully GDPR compliant" and "trusted by hundreds of companies." They are reluctant to share technical documentation or details of their conformity assessment. They want you to sign within the week.

The question: Red flags, how many can you spot?

Analysis: Multiple red flags here. GDPR compliance does not equal EU AI Act compliance, these are separate frameworks. "Trusted by hundreds of companies" is a marketing claim, not evidence of compliance. Reluctance to share documentation is a serious concern, compliant providers should be able to share this. And artificial deadline pressure is a classic sales tactic that should be resisted. This procurement should not proceed until documentation is provided and reviewed. If the vendor refuses, that tells you something important.

Scenario 4: The uncomfortable output

An AI tool your team uses is producing racially skewed outcomes

Scenario: You notice that an AI tool your team uses to prioritise customer service calls is consistently deprioritising calls from postcodes associated with minority ethnic communities. The pattern isn't obvious, you only spotted it because you looked at the data carefully.

The question: What do you do?

Analysis: This is a live bias issue with potential discrimination implications. Do not ignore it. Do not wait for someone else to notice. Document the pattern you've found and escalate immediately to your compliance team. The system should be suspended for the affected use case while the issue is investigated. This is exactly the kind of monitoring and escalation behaviour the Act expects of deployers. You have done something important by looking at the data critically, that's human oversight working as it should.

An ethical framework for AI decisions

When you're unsure, ask these questions

• Would I be comfortable if this decision were explained to the person it affects?

• Would a reasonable person consider this fair?

• Could this decision be defended to a regulator?

• Does this use AI in the way it was intended and assessed for?

• Would I be comfortable seeing this reported on the front page of a newspaper?

When you face an ambiguous AI situation, a simple ethical framework helps. Would you be comfortable explaining this decision to the person it affects? Would a reasonable person consider it fair? Could you defend it to a regulator? Is the AI being used the way it was designed and assessed for? And the classic test: would you be comfortable seeing this in a news story? If any of these questions makes you hesitate, that hesitation is worth listening to.

Learning Objectives

By the end of this chapter, you will be able to:

1. Identify how the EU AI Act is enforced and by whom

2. Explain what AI regulatory sandboxes are and who can benefit

3. Understand how enforcement decisions will shape compliance expectations

Who enforces the AI Act, and what does that mean for you?

Understanding enforcement is as important as understanding the rules. In this chapter, we'll look at how the EU AI Act is enforced, who the key authorities are, and how regulatory sandboxes offer opportunities for innovation within the law.

National market surveillance authorities

Enforcement happens at national level

· Each EU member state designates a national market surveillance authority (NMSA)

· NMSAs investigate complaints, conduct audits, and impose sanctions

· EU AI Office oversees GPAI models at EU level

· Cross-border cases coordinated between NMSAs

· Organisations operating across multiple EU states must engage with multiple authorities

The EU AI Act is enforced primarily at national level. Each EU member state designates a national market surveillance authority, this might be an existing data protection authority, a communications regulator, or a new dedicated body. For general purpose AI models, a new EU AI Office operates at EU level. If your organisation operates in multiple EU member states, you may need to engage with authorities in each of those countries, though coordination mechanisms exist for cross-border cases.

How enforcement works

Regulators can investigate, audit, and fine

· Market surveillance: authorities can inspect and audit AI systems

· Complaint-based investigations: triggered by individuals or organisations

· Corrective measures: withdrawal from market, modification, suspension of use

· Financial penalties as previously discussed

· Naming and shaming: publicising non-compliance is also a tool

National authorities have real powers. They can conduct proactive market surveillance, auditing AI systems without waiting for a complaint. Complaints from individuals or organisations can trigger investigations. Authorities can require systems to be modified, suspended, or withdrawn from the market. Financial penalties apply, as we discussed in Chapter 13. And regulators can publish findings, the reputational exposure of a public enforcement decision should not be underestimated.

AI regulatory sandboxes

Sandboxes allow supervised innovation within a safe space

· Regulatory sandboxes allow organisations to test AI in controlled conditions

· Sandbox participants get regulatory guidance and limited liability protection

· Aimed particularly at SMEs and innovative use cases

· National authorities will establish sandboxes, some are already being developed

· Participation requires a formal application and engagement with the authority

One of the less-discussed but genuinely useful provisions of the Act is the regulatory sandbox. Sandboxes allow organisations, particularly smaller businesses and innovative startups, to test novel AI systems in a controlled environment with regulatory oversight. In exchange, participants get guidance from regulators and a degree of protection from liability while testing. If your organisation is developing something genuinely new, a sandbox may be a better path than trying to assess compliance entirely in isolation.

Learning Objectives

By the end of this chapter, you will be able to:

1. Apply everything from the course to a realistic scenario

2. Complete a structured gap analysis for a hypothetical organisation

3. Demonstrate competence across the core requirements of the EU AI Act

Putting it all together, your capstone assessment

You've made it to the final assessed chapter. This capstone is designed to test whether you can apply the EU AI Act in a realistic context. We'll work through a case study organisation, TalentBridge, a fictional HR technology company, and you'll answer questions at each stage. Take your time. Think through each question carefully before selecting your answer.

Introducing TalentBridge

About TalentBridge

Background: TalentBridge is a mid-sized HR technology company based in the Netherlands that provides recruitment software to clients across the EU. Their flagship product, "HireIQ," uses machine learning to screen CVs, rank candidates, and provide shortlists to hiring managers. They also offer an integrated chatbot for candidate queries. They are considering adding a sentiment analysis feature to assess candidate confidence during video interviews.

TalentBridge provides AI-powered hiring tools to employers across the EU. As you listen to the following questions, think about what role TalentBridge plays, what their obligations are, and where the risks and red flags are.

Capstone question set

Q1. What role does TalentBridge hold in relation to HireIQ?

• a) Deployer only, they use it internally

• b) Provider ✓, they build and make the system available to other organisations

• c) Both provider and deployer

• d) Importer

Q2. What risk category does HireIQ fall into?

• a) Minimal risk

• b) Limited risk

• c) High risk ✓, it is used in employment and recruitment decisions (Annex III)

• d) Unacceptable risk

Q3. TalentBridge's clients (the employers) are:

• a) Providers of HireIQ

• b) Deployers of HireIQ ✓, they use it in their hiring processes

• c) Neither, they have no obligations under the Act

• d) Regulators

Q4. The proposed sentiment analysis feature, assessing candidate confidence during video interviews, raises which concern?

• a) It will make the software more expensive

• b) It is a prohibited use under Article 5, emotion inference in recruitment contexts ✓

• c) It will require a minor update to the privacy policy

• d) It will make the chatbot slower

Q5. TalentBridge's integrated chatbot for candidate queries falls into which risk category and what is the main obligation?

• a) High risk, requires full conformity assessment

• b) Minimal risk, no obligations

• c) Limited risk, candidates must be informed they are talking to AI ✓

• d) Unacceptable risk, chatbots in recruitment are banned

Gap analysis exercise

Identify the gaps for TalentBridge

Exercise (reflection, no single correct answer): Based on what you know about TalentBridge, identify three compliance gaps they need to address before the Act's high-risk obligations fully apply. Consider: documentation, testing, governance, and the proposed sentiment feature.

Model answer guidance:

1. The sentiment analysis feature must be abandoned or fundamentally redesigned, as described, it likely constitutes prohibited emotion inference in a recruitment context.

2. HireIQ requires a conformity assessment, comprehensive technical documentation, and registration in the EU AI database.

3. TalentBridge must ensure their deployer clients (employers) have appropriate human oversight in place and provide them with suitable instructions for use.

4. Bias testing against protected characteristics must be conducted and documented before deployment or continued use.

Well done for working through TalentBridge. The real-world application of these rules is rarely clean, you'll often face situations where the facts are ambiguous, the technology is evolving, and the commercial pressures are real. Your job is to apply sound judgment, document your reasoning, escalate when uncertain, and maintain the human oversight that sits at the heart of the EU AI Act.

Learning Objectives

By the end of this chapter, you will be able to:

1. Identify what is still being developed under the Act

2. Explain how your organisation should stay ahead of regulatory change

3. Understand how the Act may evolve over time

What's coming next, future phases and evolving obligations

The EU AI Act is not a static document. Guidance is still being developed, Annex lists can be amended, and the regulatory landscape will continue to evolve. In this final content chapter, we'll look at what's still coming and how to stay ahead.

What is still being developed

Several important elements are still in progress

• Detailed technical standards (via CEN/CENELEC and ETSI)

• Codes of practice for GPAI model providers

• Guidance on specific high-risk categories

• National authority structures in some member states

• Regulatory sandbox programmes

While the core law is in force, much of the detailed implementation guidance is still being developed. Technical standards bodies are working on the standards that will define what a conformity assessment looks like in practice. Codes of practice for GPAI providers are being developed with industry input. And national authorities are still being set up in some member states. This means some compliance questions don't yet have definitive answers, your legal team needs to stay engaged with these developments as they emerge.

How the Act may evolve

The Act includes mechanisms for amendment and adaptation

• Annex III (high-risk list) can be amended by the Commission as AI evolves

• Reviews are built into the Act at regular intervals

• Enforcement decisions will shape how rules are interpreted in practice

• International AI governance frameworks may influence EU approaches

The Act is designed to evolve with technology. The list of high-risk AI domains in Annex III can be amended by the European Commission, meaning a system that is minimal risk today could become high risk in future. Regular review cycles are built in. And as national authorities issue enforcement decisions, those decisions will clarify how ambiguous rules are interpreted. Keep an eye on enforcement activity, early cases will set important precedents.

How to stay ahead

Build adaptability into your compliance programme

• Subscribe to updates from your national authority and the EU AI Office

• Engage with industry associations and sector guidance

• Schedule annual reviews of your AI governance programme

• Maintain your AI inventory as a living document

• Build flexibility into vendor contracts to accommodate regulatory change

The organisations that will manage AI regulation most effectively are those that build adaptability into their compliance programme from the start. That means treating your AI inventory as a living document, scheduling regular governance reviews, and staying connected to regulatory developments. Industry associations and sector bodies can be useful sources of practical guidance. And your vendor contracts should include provisions for regulatory change, so that when requirements evolve, you're not locked into terms that create compliance risk.

Learning Objectives

This chapter provides a durable reference that learners can return to at any time.

Quick reference, key terms, articles, and resources

This final chapter is your reference pack. Bookmark it and come back to it whenever you need a quick reminder of a key term, a specific article, or where to find more guidance. You don't need to memorise this, just know it's here.

Key defined terms

Term

Definition

AI system

A machine-based system that infers from inputs to generate outputs such as predictions, recommendations, decisions, or content

Provider

An organisation that develops or places an AI system on the market

Deployer

An organisation that uses an AI system in a professional context

High-risk AI

An AI system listed in Annex III, subject to full compliance obligations

GPAI model

A general purpose AI model capable of performing a wide range of tasks

Conformity assessment

The process of verifying that a high-risk AI system meets Act requirements

Post-market monitoring

Ongoing assessment of a deployed AI system's safety and performance

Systemic risk

Designation for the most powerful GPAI models with additional obligations

Key articles at a glance

Article

Subject

Article 4

AI literacy obligations

Article 5

Prohibited AI practices

Articles 8–15

Obligations for high-risk AI systems

Article 13

Transparency and instructions for use

Article 14

Human oversight

Article 50

Transparency for certain AI systems (chatbots, deepfakes)

Title VIII

General purpose AI models

Annex I

Definition of AI systems

Annex III

List of high-risk AI application domains

Key resources

• EU AI Act full text: EUR-Lex (search "Regulation 2024/1689")

• EU AI Office: digital-strategy.ec.europa.eu/ai

• Your national market surveillance authority: [insert country-specific link]

• EDPB guidance on AI and GDPR: edpb.europa.eu

• ISO/IEC AI standards: iso.org (search AI standards)

• This organisation's AI usage policy: [insert internal link]

• AI compliance escalation contact: [insert internal contact]

Course completion

You've completed the EU AI Act training programme

Congratulations on completing the EU AI Act training. You've covered the foundations, the obligations, the implementation steps, and the applied learning that will help you navigate AI compliance in your day-to-day role. The law is evolving, stay curious, stay current, and remember: the goal of this regulation is not to slow down AI, but to make sure it works for people. That's something worth getting right.