AI Engineering glossary

Every buzzword the course explains, in one plain sentence. Search for a word you just heard, or pick a unit to see the terms it teaches. Each unit tag takes you to that unit on the course page.

114 terms

A

A2A: Agent-to-agent protocol: a standard for agents to call other agents.Unit 6: Agents and tool use
Agent: A model in a loop that picks tools, acts, observes the result and repeats until done.Unit 6: Agents and tool use
Agent harness: The code around a model that runs the agent loop, tools, permissions and memory.Unit 6: Agents and tool use
Agent skills: Packaged instructions and files an agent loads when a task needs them.Unit 6: Agents and tool use
Agentic: Describes a system where the model decides its own next steps.Unit 6: Agents and tool use
Agentic RAG: Retrieval offered as a tool the model calls when it needs it.Unit 5: Retrieval-augmented generation (RAG)
Attention: The mechanism that lets each token weigh every other token when computing its meaning.Unit 1: How LLMs work, intuitively Under the hood: Under the hood
AWQ / GPTQ / GGUF: Popular ways and file formats for quantizing model weights.Unit 13: Inference and serving

B

Base vs instruct model: A raw pretrained model vs one tuned to follow instructions and chat.Unit 1: How LLMs work, intuitively
Benchmark: A public test set used to compare models, such as MMLU or SWE-bench.Unit 8: Evaluation
BM25: The classic keyword-ranking formula behind most search engines.Unit 4: Embeddings and vector search

C

Chain-of-thought: A model reasoning step by step before answering, now built into reasoning models.Unit 3: Prompt engineering
Chat template: The exact format that turns a list of messages into the tokens a model sees.Unit 2: Working with model APIs
Chunk overlap: Repeating a little text between neighbouring chunks so ideas aren't cut in half.Unit 5: Retrieval-augmented generation (RAG)
Chunking: Splitting documents into pieces small enough to embed and retrieve.Unit 5: Retrieval-augmented generation (RAG)
Coding agent: An agent that reads a codebase, edits files and runs tests.Unit 6: Agents and tool use
ColBERT / late interaction: Retrieval that compares every query token with every document token for sharper matches.Unit 4: Embeddings and vector search
Compaction: Summarising a long session's history so work can continue in a fresh context.Unit 7: Context engineering
Constitutional AI: Training a model to follow a written set of principles, using AI feedback.Unit 12: Customising models
Constrained decoding / JSON mode: Restricting which tokens a model may pick so output always matches a format.Unit 2: Working with model APIs
Contamination: When benchmark questions leaked into training data, inflating scores.Unit 8: Evaluation
Context engineering: Deciding what goes into the model's context at every step of a task.Unit 7: Context engineering
Context rot: Quality dropping as the context fills with more and older material.Unit 7: Context engineering
Context window: The maximum tokens a model can read and write in one request.Unit 1: How LLMs work, intuitively
Contextual retrieval: Adding a short description of the source to each chunk before embedding it.Unit 5: Retrieval-augmented generation (RAG)
Continuous batching: Adding new requests to a running GPU batch as others finish.Unit 13: Inference and serving
Cosine similarity: A score for how closely two vectors point the same way.Unit 4: Embeddings and vector search

D

Data exfiltration: Tricking an AI system into sending private data to an attacker, for example through a link.Unit 10: Safety, security and guardrails
Diffusion model: A model that generates images by removing noise step by step.Unit 14: Beyond text
Distillation: Training a small model to imitate a larger one.Unit 12: Customising models
DPO: Direct Preference Optimization: tuning a model on pairs of better and worse answers.Unit 12: Customising models
Drift: Production inputs or model behaviour slowly changing from what you tested.Unit 11: LLMOps and observability
DSPy / prompt optimisation: Tools that rewrite prompts automatically to score better on an eval.Unit 3: Prompt engineering

E

Embedding: A list of numbers that captures meaning, so similar things sit close together.Unit 4: Embeddings and vector search
Error analysis: Reading real outputs and sorting failures into types before measuring anything.Unit 3: Prompt engineering Unit 8: Evaluation
Eval: A repeatable test of an AI system's output quality.Unit 3: Prompt engineering Unit 8: Evaluation

F

Faithfulness / groundedness: Whether an answer is actually supported by the sources it was given.Unit 5: Retrieval-augmented generation (RAG)
Few-shot: Putting a few worked examples in the prompt.Unit 3: Prompt engineering
Fine-tuning: Further training a model on your own examples.Unit 12: Customising models
Foundation model: A large model pretrained on broad data and adapted to many tasks.Unit 0: Start here Unit 1: How LLMs work, intuitively
FP8 / INT4: Lower-precision number formats that shrink models and speed up inference.Unit 13: Inference and serving
Function calling: Another name for tool calling.Unit 2: Working with model APIs

G

Golden set: A small, trusted set of inputs with known good answers.Unit 3: Prompt engineering Unit 8: Evaluation
Grounding: Tying answers to provided sources so they can be checked.Unit 5: Retrieval-augmented generation (RAG)
GRPO: A reinforcement learning method that scores groups of answers against each other.Unit 12: Customising models
Guardrails: Checks around a model that block unsafe inputs or outputs.Unit 10: Safety, security and guardrails

H

Hallucination: Confident output that isn't true or isn't supported by the sources.Unit 1: How LLMs work, intuitively
HNSW: A graph-based index for fast approximate nearest-neighbour search.Unit 4: Embeddings and vector search
Human in the loop: A person approving or correcting an AI system's actions at set points.Unit 6: Agents and tool use
Hybrid search: Combining keyword and vector search results.Unit 4: Embeddings and vector search
HyDE: Retrieving with an embedding of an imagined answer instead of the question.Unit 5: Retrieval-augmented generation (RAG)

I

Inference: Running a trained model to get outputs.Unit 13: Inference and serving

J

Jailbreak: A prompt crafted to make a model ignore its safety rules.Unit 10: Safety, security and guardrails

K

Knowledge cutoff: The date after which a model has no training data.Unit 1: How LLMs work, intuitively
KV cache: Stored attention keys and values that make generating each new token cheaper.Unit 13: Inference and serving

L

Latency (TTFT, TPOT): Time to first token, and time per output token.Unit 13: Inference and serving
Lethal trifecta: Private data, untrusted content and a way to send data out, together in one agent.Unit 10: Safety, security and guardrails
LLM: Large language model: a transformer trained on huge amounts of text.Unit 1: How LLMs work, intuitively
LLM gateway: A proxy in front of model providers that handles keys, quotas, routing and logs.Unit 11: LLMOps and observability
LLM-as-a-judge: Using a model to grade another model's output against a rubric.Unit 8: Evaluation
LLMOps: Practices and tools for running LLM features in production.Unit 11: LLMOps and observability
Logits / logprobs: The model's raw scores and probabilities for each possible next token.Unit 1: How LLMs work, intuitively
LoRA / QLoRA: Fine-tuning by training small add-on matrices instead of all weights.Unit 12: Customising models
Loss, epoch, overfitting: How wrong a model is, one pass over the data, and memorising instead of learning.Unit 12: Customising models Under the hood: Under the hood

M

Max tokens / stop sequences: Limits that end a response at a length or at chosen text.Unit 2: Working with model APIs
MCP: Model Context Protocol: an open standard for connecting models to tools and data.Unit 6: Agents and tool use
Mixture of experts (MoE): A model where each token uses only a few of many sub-networks.Under the hood: Under the hood
Model / system card: The document a lab publishes about a model's abilities, limits and safety testing.Unit 16: Staying expert
Model routing: Sending each request to the cheapest model that can handle it.Unit 11: LLMOps and observability
Multimodal: Handling more than one kind of input or output, like text plus images.Unit 1: How LLMs work, intuitively Unit 14: Beyond text

O

On-device model: A model small enough to run on a phone, laptop or browser.Unit 14: Beyond text
Open-weight vs open-source: Downloadable weights vs a licence that also allows full reuse, data and code.Unit 1: How LLMs work, intuitively
OpenTelemetry / span: The standard for traces, where each step of a request is one span.Unit 11: LLMOps and observability
Orchestration frameworks: Libraries for chaining model calls, RAG and agents, such as LangChain or agent SDKs.Unit 9: Building AI products end to end

P

Parameters: The learned numbers inside a model; "70B" means 70 billion of them.Unit 1: How LLMs work, intuitively
pass@k: The chance that at least one of k attempts succeeds.Unit 8: Evaluation
PEFT: Parameter-efficient fine-tuning, the family LoRA belongs to.Unit 12: Customising models
PII redaction: Removing personal data such as names and emails from prompts and logs.Unit 11: LLMOps and observability
Prefill vs decode: Reading the whole prompt at once, then generating output one token at a time.Unit 13: Inference and serving
Pretraining: The first, huge training run that teaches a model language and knowledge.Unit 1: How LLMs work, intuitively
Prompt caching: Reusing the processed form of a repeated prompt prefix to save cost and time.Unit 2: Working with model APIs
Prompt injection: Untrusted text that hijacks a model's instructions.Unit 3: Prompt engineering Unit 10: Safety, security and guardrails

Q

Quantization: Storing weights with fewer bits to save memory and speed up inference.Unit 13: Inference and serving

R

RAG: Retrieval-augmented generation: fetching relevant data and adding it to the prompt.Unit 5: Retrieval-augmented generation (RAG)
ReAct: The reason-then-act loop that most agents use.Unit 6: Agents and tool use
Reasoning effort / thinking budget: A setting for how long a reasoning model may think before answering.Unit 2: Working with model APIs
Reasoning model: A model trained to think at length before answering, the default in 2026.Unit 1: How LLMs work, intuitively
Recall@k, MRR, nDCG: Scores for how well retrieval puts the right documents near the top.Unit 4: Embeddings and vector search
Reranker: A model that re-scores retrieved results for relevance.Unit 4: Embeddings and vector search
RFT: Reinforcement fine-tuning: provider APIs that tune a model against your grader.Unit 12: Customising models
RLHF: Reinforcement learning from human feedback, used to make models helpful and safe.Unit 1: How LLMs work, intuitively Unit 12: Customising models
RLVR: Reinforcement learning from verifiable rewards, like tests that pass or answers that check out.Unit 12: Customising models

S

Sandbox: An isolated environment where an agent can run code without touching real systems.Unit 6: Agents and tool use Unit 10: Safety, security and guardrails
Scaling laws: Predictable gains from more compute, data and parameters.Under the hood: Under the hood
Semantic caching: Reusing an earlier answer when a new question means the same thing.Unit 11: LLMOps and observability
SLM: Small language model, roughly under 10 billion parameters.Unit 14: Beyond text
Speculative decoding: A small model drafts tokens that a big model checks in one pass.Unit 13: Inference and serving
Structured outputs: Forcing model output to match a JSON schema.Unit 2: Working with model APIs
Subagent: A helper agent started by another agent for one piece of work, with its own context.Unit 6: Agents and tool use
Synthetic data: Training or test data generated by models.Unit 8: Evaluation Unit 12: Customising models
System prompt: Standing instructions that set a model's role and rules.Unit 2: Working with model APIs Unit 3: Prompt engineering

T

Temperature: A setting that makes output more random (high) or more predictable (low).Unit 1: How LLMs work, intuitively
Tensor parallelism: Splitting each layer of a model across several GPUs.Unit 13: Inference and serving
Test-time compute: Spending more computation while answering, such as longer thinking.Unit 1: How LLMs work, intuitively Under the hood: Under the hood
Throughput / goodput: Tokens served per second, and the share that met the latency target.Unit 13: Inference and serving
Token: The unit of text a model reads and writes, often a piece of a word.Unit 1: How LLMs work, intuitively
Tool calling: A model asking your code to run a function with given arguments.Unit 2: Working with model APIs Unit 6: Agents and tool use
Tool poisoning: Hidden instructions in a tool's description or output that hijack an agent.Unit 10: Safety, security and guardrails
Top-p / top-k: Sampling settings that limit which next tokens are considered.Unit 1: How LLMs work, intuitively
Transformer: The neural network architecture behind modern LLMs.Unit 1: How LLMs work, intuitively Under the hood: Under the hood

V

Vector database: A database built to store embeddings and search them by similarity.Unit 4: Embeddings and vector search
Vibe coding: Building software by prompting an AI and accepting its code with little review.Unit 0: Start here
vLLM: A popular open-source engine for serving LLMs fast.Unit 13: Inference and serving

Z

Zero-shot: Asking a model to do a task with no examples.Unit 3: Prompt engineering