AI Engineering

What is AI engineering?

How it differs from ML engineering, data science and research, and what companies hire AI engineers to do

The AI stack

Models, APIs, orchestration, retrieval, evals, serving: the layers and who builds each

AI, ML, deep learning, generative AI

The nested terms untangled, with a short history from rules to transformers

Python for AI engineers

HTTP and JSON, environment variables and secrets, async calls, and project setup with uv

Your first API call

Send a prompt, read the reply, and see tokens and cost on your own screen

Your toolkit

Notebooks, a local model with Ollama, and how to run the course labs

Working with coding agents

Pairing with AI coding tools, reviewing AI-written code, and spec-first development

Tokens and tokenization

Why models see tokens, not words, and what that does to cost, limits and spelling bugs

Next-token prediction

How a model writes one token at a time, logits and probabilities, and why that explains most of its behaviour

Transformers and attention

The architecture in pictures: embeddings, attention, layers, without the maths

How models are trained

Pretraining, then instruction and preference tuning: base vs instruct models (details in Unit 12)

Sampling: temperature, top-p, top-k

How the next token is picked, and which settings to use for which job

The context window and its limits

What fits, what it costs, knowledge cutoffs, hallucinations, and why long context isn't free

Reasoning models and the model landscape

Reasoning models as the 2026 default, open-weight vs closed, sizes, families, and multimodal in one note

Messages, roles and parameters

System, user and assistant turns, chat templates, stop sequences and max tokens

Streaming

Server-sent events, time to first token, and streaming UIs

Tokens, pricing and limits

Input vs output pricing, rate limits, quotas, and estimating a feature's cost

Reasoning controls

Effort levels and thinking budgets, and what they do to quality, latency and cost

Structured outputs

JSON mode, schemas, constrained decoding, and validating what comes back

Tool and function calling

Letting a model ask your code to act, and the request-response loop that makes it work

Images, documents and audio as input

Vision, PDFs, transcription, and their token costs

Realtime and voice APIs

Speech-to-speech sessions, turn-taking basics, and when voice is worth it (deep dive in Unit 14)

Prompt caching and batch APIs

Cutting cost and latency for repeated prefixes and offline jobs

Choosing a model

Quality, latency and cost trade-offs, routing, and fallbacks

Build: a CLI assistant

A streaming chat tool with tools and cost tracking

Anatomy of a good prompt

Clear tasks, context, constraints, format, and the system prompt

Examples: zero-shot and few-shot

When examples help, how many, and how to pick them

Reasoning vs non-reasoning models

When to pay for a reasoning model, and why "think step by step" is now mostly legacy

Structure with tags and delimiters

XML tags, sections, and separating instructions from data

Prompt chaining

Splitting a task into steps, and passing results between them

Prompt templates and versioning

Treating prompts as code: variables, tests and history

Your first eval eval

Error analysis on real outputs, a small golden set, and pass rates you can compare

Prompt optimisation

Letting tools like DSPy and GEPA tune prompts against your eval

Build: a structured extractor

Turning messy emails into validated JSON, with a test set

What embeddings are

Turning text into vectors where close means similar

Measuring similarity

Cosine similarity, dot product, distance, and normalisation

Choosing an embedding model

Dimensions, languages, benchmarks (MTEB), cost, and matryoshka embeddings

Vector databases

pgvector, Pinecone, Qdrant, Weaviate and friends, and what they add over an array

Approximate nearest neighbour indexes

HNSW and the recall vs speed dial (IVF and PQ in "Go deeper")

Keyword and hybrid search

BM25, why keywords still matter, and combining scores (reciprocal rank fusion)

Reranking and late interaction

Cross-encoders, rerank APIs and ColBERT-style retrieval for a better top 5

Measuring retrieval eval

Recall@k, MRR and nDCG on a labelled query set

Build: semantic search

Search over a docs site, with filters, a reranker and a retrieval eval

Why RAG

Fresh and private knowledge without retraining (the full prompt vs RAG vs fine-tune guide is in Unit 12)

Ingestion: parsing and cleaning

HTML, tables and messy sources, and why garbage in is the top RAG failure

Parsing PDFs with vision models

Vision-based document parsing and ColPali-style retrieval, which have replaced most OCR pipelines

Chunking strategies

Fixed, recursive, semantic and structure-aware chunks, overlap and size

The retrieve-then-generate loop

Putting retrieved context into the prompt, with citations

Better queries

Query rewriting, multi-query, HyDE, and decomposing hard questions

Better context

Parent-child chunks, contextual retrieval, metadata and freshness

Agentic RAG

Retrieval as a tool the model calls when it needs it, now the default pattern, plus text-to-SQL and a short look at graph RAG

Faithfulness and groundedness eval

Checking answers against sources, citation accuracy, and why RAG fails

Case study: a docs Q&A assistant

Design, estimates and trade-offs for a company knowledge bot

From chatbot to agent

The agent loop: think, act, observe, repeat (ReAct)

Designing tools

Names, descriptions, parameters and errors a model can actually use

Workflows vs agents

Fixed pipelines, routers, orchestrator-workers, and when autonomy pays

Planning and reflection

Plans, sub-goals, self-correction and stopping conditions

Model Context Protocol (MCP)

The standard way to plug tools and data into any model: transports, auth, and agent-to-agent (A2A)

Agent harnesses and SDKs

The loop as a product: subagents, skills, sandboxes, checkpoints and agent SDKs

Long-running and coding agents

How coding agents read repos, run tests and work for hours without losing the plot

Computer and browser use

Agents that click, type and read screens

Human in the loop

Approvals, escalation, and designing where a person must say yes

Multi-agent systems, with caution

Specialists and handoffs, their costs, and why to start with one agent

Task success eval

Measuring agents by outcome, trajectory, tool-call accuracy and cost per task

Case study: a customer support agent

Tools, escalation, evals and failure handling

From prompts to context

Why the 2026 skill is managing the whole context, not one clever prompt

The context budget and context rot

How quality drops as context fills, and how to spend tokens on purpose

Just-in-time retrieval

Letting the agent fetch files, docs and data when needed instead of stuffing them in up front

Trimming tool results

Summaries, pagination and filters so tool output doesn't drown the task

Compaction and summarisation

Keeping long sessions going by compressing history without losing decisions

Memory and notes

Memory files, scratchpads and long-term stores, and what to forget

Build: a long-running research agent

An agent that works through a big task across many context windows

Why evals are the job

Vibes vs measurement, and the eval-driven development loop

Error analysis in depth

Reading traces, labelling failure modes, and deciding what to measure

Building a test set

Golden examples, edge cases, synthetic data, and how many you need

Code-based checks

Exact match, regex, schema checks, unit tests for outputs

LLM as a judge

Rubrics, pairwise comparison, judge bias, and calibrating against humans

Evaluating RAG and agents end to end

Combining the unit-level evals into one suite, with pass@k for agents

Benchmarks and leaderboards

MMLU, GPQA, SWE-bench and others, contamination, saturation, and reading them sceptically

Evals in CI

Regression suites that run on every prompt or model change

Architecture of an LLM app

Front end, API, orchestration, retrieval, models, storage and queues

Orchestration frameworks and agent SDKs

LangChain, LlamaIndex, Vercel AI SDK, agent SDKs and plain code, compared

Conversation state

Sessions, history trimming, summarising and storage

Latency

Streaming, parallel calls, smaller models and UX that hides waits

Cost control

Token budgets, exact caching, and cheaper models (semantic caching and routing in Unit 11)

Reliability

Retries, timeouts, fallbacks across providers, and rate limits

UX for AI features

Showing uncertainty, citations, feedback buttons, and undo

Build: a full-stack AI app

A deployed RAG chat app with auth, streaming, evals and cost tracking

The threat model

OWASP Top 10 for LLM applications, mapped to real incidents

Prompt injection, direct and indirect

Attacks through user input, documents, web pages and tool results

The lethal trifecta

Private data plus untrusted content plus a way to send data out, and how to break the triangle

Tool and MCP supply-chain attacks

Tool poisoning, rug pulls, and data exfiltration through markdown images and links

Jailbreaks and red teaming

How guardrails get bypassed, and how teams test for it

Data leakage and privacy

PII, secrets in prompts, training on user data, and retention

Guardrails

Input and output filters, moderation models, allow-lists and policy checks

Sandboxing and agent identity

Sandboxed code execution, scoped tokens and OAuth for agents, and least privilege

Responsible AI

Bias, fairness, transparency, and regulation such as the EU AI Act (dated, as its timeline is still moving)

Tracing LLM calls

Spans for prompts, retrievals and tool calls, and the tools that collect them

OpenTelemetry for GenAI

The standard span conventions, so traces work across vendors

LLM gateways and routing

One front door for many providers: keys, quotas, model routing and semantic caching

Monitoring quality and drift

Online eval scores, drift, and alerting

User feedback and annotation queues

Collecting thumbs and comments, and turning them into labelled data

Prompt and model management

Versioning, rollouts, model upgrades, deprecations and migrations

Cost and usage dashboards

Per-feature and per-customer spend, and anomaly alerts

PII in logs and retention

What to store, what to redact, and for how long

The data flywheel

Turning production traces into test sets and training data

Incidents

When a model regresses or a provider goes down: playbooks and fallbacks

Prompt, RAG or fine-tune?

The one decision guide, by problem type, data and budget

GPU and memory basics for training

Just enough about VRAM and memory math to plan a fine-tune (more in Unit 13)

Data quality and curation

Where fine-tunes really succeed or fail: collecting, cleaning and balancing examples

Supervised fine-tuning

Datasets, formats, epochs, overfitting and evaluation

LoRA, QLoRA and PEFT

Training a small adapter instead of the whole model

Preference tuning with DPO

Tuning on better and worse answer pairs, with RLHF as the concept behind it

RL fine-tuning

RLVR, GRPO, reinforcement fine-tuning APIs, and designing rewards

Distillation

Teaching a small, cheap model to copy a big one

Synthetic data

Generating, filtering and de-duplicating training data

Build: fine-tune a small model

A LoRA fine-tune that beats a prompt on one narrow task, with evals

GPUs for engineers

Memory, bandwidth, FLOPs, and why LLM inference is memory-bound

Memory math

Weights, the KV cache, and sizing a GPU for a model and context length

Prefill and decode

The two phases of inference, throughput vs goodput

Batching

Static vs continuous batching, and throughput vs latency

Quantization

FP8, INT8, INT4, GGUF, AWQ and GPTQ, and what they cost in quality

Speeding up decoding

Speculative decoding, prefix caching, paged attention and FlashAttention

Serving stacks

vLLM, SGLang, TensorRT-LLM, llama.cpp and Ollama, managed endpoints, and TGI as legacy

Parallelism

Tensor and pipeline parallelism, and when a model needs more than one GPU

Latency metrics

Time to first token, tokens per second, p99, and SLOs for AI

Self-host or API?

Cost per million tokens, utilisation, privacy, and the break-even point

Vision-language models

Image understanding and document AI

Voice agents in depth

Speech pipelines vs speech-to-speech models, turn-taking and sub-second latency

Image and video generation

Diffusion models, prompts, control and editing

Multimodal retrieval

Image and audio embeddings, and searching across media

Small and on-device models

SLMs, edge and WebGPU, and when a 3 to 8B model is enough

How to approach an AI design question

Requirements, estimates, model choice, evals and risks

Design a ChatGPT-style chat service

Streaming at scale, history, rate limits, cost per user

Design enterprise search with RAG

Permissions-aware retrieval across many data sources

Design an AI coding assistant

Context gathering, latency budgets, and evals on real repos

Design a deep-research agent

Planning, parallel search, source tracking and long-running context

Design an agentic workflow with approvals

Multi-step automation with human sign-off and audit trails

Design a voice agent

Speech pipeline, turn-taking and sub-second latency

Design an AI content moderation system

Classifiers plus LLMs, queues, humans in the loop, and appeals

Reading AI papers

A three-pass method, and where to find the papers that matter

Evaluating new models on your evals

Your own suite as the yardstick, not the leaderboard

The frontier

What's moving now: reasoning, long context, agents and on-device models

Building a portfolio

Projects that show judgement, and writing about them

Career paths and interviews

AI engineer roles, system design rounds and eval take-homes

Neural networks from scratch

Neurons, layers, activations, loss, epochs and overfitting

Gradient descent and backpropagation

How a network learns, step by step

Attention, with the maths

Queries, keys, values, multi-head attention and positional encodings

Inside a transformer block

Layer norm, MLPs, residuals, and decoder-only models

Mixture of experts

Sparse models, routing, and why they're cheap to run

Scaling laws and test-time compute

Compute, data and parameters, and the newer scaling of thinking at inference time

Training at scale

Data pipelines, mixed precision, and data, tensor and pipeline parallelism