AI Architect: Enterprise AI System Design

Preview what Module 1 covers: the AI Architect role and its four responsibilities, the modern AI capability landscape, and the feasibility and scoping discipline the module builds toward.

Explain why a model is not a system, and identify the four properties (probabilistic outputs, soft failure, non-linear cost, governance exposure) that make AI components architecturally different from standard software.

Distinguish the AI Architect role from adjacent roles (ML Engineer, Data Scientist, Solutions Architect), and name the four responsibilities — feasibility, design, governance, communication — that structure the rest of the course.

Compare API-first proprietary models, self-hosted open-weight models, and fine-tuned specialised models by capability, cost, and architectural tradeoff, without needing to track which named model currently leads.

Identify the domains where classical ML still beats an LLM, recognise what makes the agent paradigm architecturally different, and apply a three-axis (capability / cost / control) framework to position an AI approach.

Assess a proposed AI initiative against five feasibility dimensions (data availability, task structure, error tolerance, latency, human-in-the-loop viability) and four red flags that mean AI is not the answer.

Classify an AI problem along the generative/discriminative and retrieval/reasoning axes, place it on the automation spectrum, and use a decision tree to choose between prompting, RAG, fine-tuning, or classical ML.

Produce a six-section AI system scoping document — problem statement, approach category, system boundary, success metrics, key unknowns, stakeholder summary — that decomposes a vague business request into buildable components.

Apply the scoping framework end to end to a realistic customer-service request, decomposing it into five distinct AI components with a defined system boundary and measurable success criteria.

Apply feasibility and scoping to a harder, latency-constrained, heavily regulated scenario, and correctly separate a fast deterministic path from a slower LLM-assisted one.

Recognise the five most common scoping and feasibility mistakes by their early "tell," and apply the low-cost prevention for each before it becomes an expensive redesign.

Apply the feasibility framework and the six-section scoping structure to a realistic business brief, entirely optional and self-assessed.

A short, dated check-in naming current examples of the frontier/mid/small/on-device tiers and the reasoning-mode cost multiplier — the numbered lessons teach tiers and dials on purpose, so this is the one place named models are allowed to appear; re-cut whenever the monthly snapshot review finds something worth updating, not on a fixed schedule.

Preview what Module 2 covers: the six core AI architecture patterns, with deep dives into RAG and agents — the two patterns that carry the most production complexity.

Name and choose among the six core AI architecture patterns — inference pipeline, RAG, agentic loop, ensemble/router, human-in-the-loop, multimodal — and recognise when a system needs two or three combined.

Build a complete RAG stack across both the ingestion pipeline (parsing, chunking, embedding, indexing) and the retrieval pipeline (query rewriting, hybrid search, reranking, generation).

Diagnose and architecturally fix coreference collapse, absence blindness, and write-side staleness — the three RAG failure modes that don't show up until real users hit real data.

Design an agent's reasoning loop, a tool registry of atomic and reversible tools, and a four-tier memory architecture, and apply zero-trust principles to multi-agent coordination.

Prevent agent cost spirals, non-determinism, and unbounded context growth with hard resource limits, and replace prompt-only safety with application-layer policy enforcement.

Produce AI architecture diagrams that make model versions, probabilistic-vs-deterministic flows, latency profiles, human review points, and evaluation hooks visible at a glance.

Apply contextual chunk enrichment, a retrieval quality gate, and event-driven re-ingestion to five RAG problems that survive a clean tutorial but break in production — and know when GraphRAG is (and isn't) worth its cost.

Defend a RAG system against indirect prompt injection from retrieved content, route structured (table/database) queries correctly instead of forcing them through vector search, and build a golden query set to calibrate retrieval thresholds.

Apply prompt/semantic/embedding caching, model-version tracking, and trajectory-based evaluation to keep an agent system reliable, cost-controlled, and auditable at production scale.

Select the right vector database (Pinecone, pgvector, Qdrant, Weaviate, or Chroma) for a given operational, cost, and data-residency context, and understand the migration cost of switching later.

Choose between HNSW and IVF (and IVF-PQ) indexing based on recall, memory footprint, and build-time tradeoffs, and configure the index type deliberately rather than by default.

Decide when to use MCP for model-to-tool integration, A2A for agent-to-agent coordination, both, or neither, based on portability and composability needs.

Select an agent runtime — graph-based, role-based, conversation-driven, or a provider SDK — based on durability/checkpointing needs, control requirements, and task shape.

Design in-context, external short-term, and long-term memory tiers matched to actual persistence requirements, and apply deliberate write gates before anything enters long-term storage.

Architect GDPR-compliant agent memory (erasure, minimisation, purpose limitation, TTLs) and choose an ownership model — isolated, shared, or orchestrator-owns-state — for multi-agent memory.

Recognise the six most common RAG and agent architecture mistakes — from naive chunking to prompt-only safety — and apply the specific fix for each.

Combine RAG, agent, and human-in-the-loop patterns into one coordinated system with an intelligent routing layer, shared infrastructure, and a conservative escalation-by-default safety net.

Recognise when a corpus and query distribution justify GraphRAG over standard RAG, and design the graph-level access control that a multi-matter legal knowledge base requires.

Design a complete RAG stack component by component for a realistic scenario, entirely optional and self-assessed.

Preview what Module 3 covers: the model selection framework, prompt engineering as architecture, fine-tuning, and the full build-vs-buy-vs-open-weight decision.

Score candidate models against five dimensions — task capability fit, latency profile, cost at scale, data privacy posture, operational complexity — and build a weighted scorecard defensible to stakeholders.

Treat the system prompt as a versioned, tested, staged-deployment artefact rather than a string constant, and recognise the specific signals that mean prompting has genuinely hit its ceiling.

Distinguish knowledge (RAG) from behaviour (fine-tuning), identify the four situations where fine-tuning adds real value, and source labeled data via synthetic generation, distillation, or a production feedback loop.

Choose between proprietary API, a managed cloud AI platform, self-hosted open-weight, and on-device deployment, and model the cost crossover point between API and self-hosted at your actual scale.

Apply the model selection framework to two regulated domains that reach the same architectural conclusion — fine-tuned, self-hosted, open-weight — through entirely different reasoning.

Recognise the six most common model selection mistakes — from fine-tuning for knowledge to linear cost extrapolation — and apply the specific fix for each.

Apply the five-dimension scorecard and cost model to defend a model selection in a written memo, entirely optional and self-assessed.

Preview what Module 4 covers: the AI integration layer, handling probabilistic outputs, AI-specific non-functional requirements, and data pipeline architecture.

Design a centralized AI gateway that provides request routing, rate limiting, cost attribution, model swapping, and caching as one controlled entry point for every AI call.

Build a three-layer validation stack (schema, semantic, business rule) for every AI output that reaches a downstream system, with a matched fallback strategy — retry, deterministic fallback, or human escalation — for each failure mode.

Design to P95/P99 latency rather than the median, apply exact-match/semantic/KV caching correctly, and build graceful degradation — feature flags, defined fallbacks, circuit breakers — so an AI outage never breaks the whole product.

Build a six-stage ingestion pipeline, choose a freshness strategy (scheduled, incremental, or event-driven), and design the data lineage record that debugging, GDPR erasure, and regulatory audit all depend on.

Integrate AI into a legacy monolith via a sidecar architecture without modifying it, and design a shared AI gateway with per-feature isolation for a multi-team cloud-native platform.

Recognise the six most common integration mistakes — from embedding AI calls directly in application code to ignoring tail latency — and apply the specific fix for each.

Configure LiteLLM as a production AI gateway: model aliasing for swap-safe routing, metadata-based cost attribution, per-service rate limits, and TTL-based caching.

Design a complete integration architecture with a validation and fallback strategy for a realistic scenario, entirely optional and self-assessed.

Preview what Module 5 covers: responsible AI as an architectural discipline, AI-specific security, the OWASP Agentic AI Top 10, guardrails, observability, and the regulatory landscape.

Translate the six responsible-AI pillars — fairness, reliability, privacy, transparency, accountability, inclusiveness — into concrete architectural constraints, and design human oversight checkpoints for autonomous agent actions.

Defend against direct and indirect prompt injection, close data-leakage vectors (memorisation, RAG over-retrieval, log leakage), and design document-level access control that filters at query time, not after.

Map each of the ten OWASP Agentic AI risks — from prompt injection to unauthorised multi-agent trust — to the specific architectural decision that mitigates it.

Turn a threat model into an executable adversarial test plan with specific inputs, pass/fail criteria, and predefined launch-gate exit criteria, using both manual and automated (garak/PyRIT) testing.

Design input guardrails (content classification, PII redaction, jailbreak detection) and output guardrails (toxicity, grounding, format validation), and place each correctly across inline, async, and offline execution modes.

Instrument every AI interaction with full-chain distributed tracing, and design tamper-evident, multi-tenant-isolated audit infrastructure that can survive a regulator's request.

Map the EU AI Act's risk tiers, the NIST AI RMF, and sector-specific rules (HIPAA, SR 11-7, GDPR) to the concrete architectural requirements they impose on a system you're designing today.

Implement a tamper-evident audit log with externally held checkpoints, prove tenant isolation continuously with adversarial tests, and separate operational logs from legal-defensibility logs.

Determine which regulation actually binds a system — the one that looks obvious isn't always the one with teeth — and design the fairness, oversight, and audit architecture it requires.

Recognise the six most consequential governance mistakes — from treating governance as a checkbox to security-testing only the model, not the system — and apply the specific fix for each.

Identify governance gaps and specify their architectural mitigations for a realistic scenario, entirely optional and self-assessed.

Preview what Module 6 covers: the three-layer measurement framework — infrastructure telemetry, LLM evaluation, and business outcome metrics — built end to end across this module.

Define pass/fail success criteria with real numbers before writing a feature, and build a golden dataset that grows as a living artefact rather than a one-off launch task.

Build a golden dataset covering happy-path, edge, failure, and adversarial cases, diagnose quality problems using the four RAGAS metrics, and gate deployments on regression in CI/CD.

Use LLM-as-judge as a calibrated trend signal (not ground truth), design a stratified human review queue, and apply the signal hierarchy when automated and human signals disagree.

Choose the right business metrics for your system type (containment rate, straight-through processing, conversion), and design the correlation ID architecture that connects AI behaviour to business outcomes.

Monitor token consumption per intent class to catch invisible cost drift, and run daily canary inputs and format-compliance tracking to catch a silent provider model update the day it happens.

Diagnose two realistic AI failures — a call-centre chatbot and a document-processing regression — using the complete three-layer measurement stack, from business metric down to root cause.

Recognise the six most common measurement blind spots — from telemetry-as-evaluation to a golden dataset that's never updated — and apply the specific fix for each.

Stand up a complete evaluation stack: Langfuse tracing and dataset promotion, RAGAS scoring with threshold calibration, and a CI/CD gate that blocks deployment on regression.

Design a complete three-layer measurement architecture for a realistic scenario, entirely optional and self-assessed.

Preview what Module 7 covers: AI-specific scaling dimensions, cost architecture, LLMOps as an operating discipline, and the production readiness checklist.

Design for AI-specific scaling dimensions — token throughput vs. request throughput, context-window memory cost, stateful session handling, streaming timeouts — and choose autoscaling signals that actually reflect AI-specific demand.

Build a cost model at three scales (launch, 10x, projected peak), apply model routing and prompt optimisation to cut inference spend, and calculate true ROI (not a benefit-cost ratio) for a caching investment.

Put prompts in version control with staged rollouts, build a prompt regression test suite that blocks bad deployments, and catch silent provider-side model drift with a canary set.

Answer all twenty questions on the production readiness checklist — spanning reliability, cost, observability, security, and operations — before a system is considered ready to go live.

Compare the real cost of retrofitting scale onto a successful beta against designing a greenfield system for scale from day one, using the same cost, routing, and LLMOps tools either way.

Recognise the six most common scaling and cost-architecture mistakes — from linear cost extrapolation to no cost circuit breakers — and apply the specific fix for each.

Design the full production architecture — scaling, cost, LLMOps, and readiness — for a realistic scenario, entirely optional and self-assessed.

Synthesize all seven modules into a complete AI architecture proposal for a realistic enterprise scenario, entirely optional and self-assessed.