150 AI terms operators actually need.

A/B testing prompts is the practice of running two prompt variants against the same eval set and comparing scores.

An adversarial input is a carefully crafted prompt or file designed to cause a model to behave incorrectly.

Agent evaluation is the practice of measuring how often an agent completes tasks correctly, efficiently, and safely.

An agent fallback is the path the system takes when the primary agent or model fails or times out.

Agent guardrails are pre- and post-checks that constrain what an agent can do, say, or spend.

An agent handoff is the moment when one agent passes control, plus accumulated context, to another agent.

The agent loop is the cycle of think, act, observe that an agent runs until it finishes a task or hits a stop condition.

Agent memory is any mechanism that lets an agent remember facts across turns or sessions beyond the raw context window.

Agent retry logic is the policy that decides when and how to repeat a failed step.

An agent swarm is a multi-agent system where many lightweight agents work in parallel on independent slices of a problem.

Agent telemetry is the logging, tracing, and metrics layer that records what an agent did and why.

An agent timeout is a hard upper bound on how long an agent run is allowed to take.

An agent token budget is the maximum number of tokens an agent run is allowed to consume across all model calls.

An AI agent is a system where an LLM can plan, call tools, and take multi-step actions toward a goal, not just answer a single prompt.

AI Ops is the operational discipline of running AI systems in production reliably, safely, and cost-effectively.

An AI policy is a written set of rules describing how an organization may build, buy, and use AI.

AI safety is the practice of preventing AI systems from causing harm, whether through misuse, error, or unintended behavior.

The AI supply chain is the full set of models, datasets, libraries, and providers that flow into a deployed AI system.

Alignment is the broad effort to make AI systems behave in ways that match human intent and values.

Attention is the mechanism that lets a model weigh how relevant every other token in the input is when generating each new token.

An audit log records every relevant AI interaction with enough detail to reconstruct what happened and who was responsible.

AutoGen is Microsoft's open-source framework for building multi-agent conversations.

Batch inference runs many inputs through a model together, trading latency for throughput.

Bias evaluation measures whether a model's behavior differs systematically across protected attributes like gender, race, or age.

BM25 is a classic keyword-based ranking function used by search engines for decades.

C2PA, the Coalition for Content Provenance and Authenticity, is the open standard for cryptographically signing media with origin metadata.

A caching layer stores prompt-response pairs or intermediate results so identical requests can be served without a model call.

A canary release rolls out a new model or prompt to a small slice of traffic first and watches for regressions before expanding.

Chain-of-thought prompting asks the model to show its reasoning before producing a final answer.

Chunking is the process of splitting documents into smaller passages before they are embedded and indexed.

CI for prompts is continuous integration adapted to language model workflows: every prompt change triggers automated evaluation.

A citation is a link or marker in the model's answer that points to the source passage it relied on.

A completion is the text a model generates in response to a prompt.

Content moderation is the process of identifying and acting on harmful or policy-violating content in AI inputs or outputs.

Content provenance is a verifiable record of where a piece of media came from, who edited it, and how.

Context compression shrinks long retrieved passages down to only the most relevant sentences before sending them to the model.

The context window is the maximum amount of text, measured in tokens, that a model can consider at once.

A copyright filter detects when model outputs reproduce copyrighted text, lyrics, or code verbatim.

Cosine similarity measures the angle between two vectors and is the standard metric for comparing embeddings.

Cost attribution links each dollar of AI spend back to the feature, team, or customer that triggered it.

Cost per token is the price you pay per 1,000 or 1,000,000 input or output tokens.

CrewAI is a framework for orchestrating role-based multi-agent teams with clear responsibilities and hand-offs.

Data poisoning is an attack where corrupted training data causes a model to behave incorrectly on specific inputs.

Data residency is the requirement that data, including prompts and responses, stay within specific geographic boundaries.

Dense retrieval uses neural embeddings to score documents against queries in a continuous vector space.

A document loader is a small piece of software that pulls source content out of PDFs, websites, databases, or SaaS apps and normalizes it.

DPO, or Direct Preference Optimization, is a simpler alternative to RLHF that fine-tunes a model directly on preference pairs without a separate reward model.

An embedding model turns text, images, or other inputs into numeric vectors that capture meaning.

Embeddings are numeric vectors that represent the meaning of a piece of text, image, or other input.

Encoder-decoder is a transformer variant where one stack reads the input and another stack generates the output.

An eval set is a curated collection of inputs and expected outputs used to measure model or prompt quality.

An evaluation pipeline runs eval sets against new prompts or models on every change and produces a scorecard.

An executor agent is the sub-agent that actually performs the steps from a plan, calling tools and reporting results.

Factuality eval scores how often a model's claims can be verified against ground-truth sources.

A fairness metric quantifies how equally a model treats different groups of users.

A fallback model is the alternate model used when the primary model fails, times out, or refuses.

Few-shot prompting is the technique of including several worked examples of input and output in the prompt to teach the model the pattern.

Fine-tuning is the process of continuing to train a base model on your own labeled dataset so it specializes in a specific task or voice.

A foundation model is a large model trained on broad data that can be adapted for many downstream tasks.

Function calling is the API feature that lets a model emit a structured request to invoke one of a developer-defined set of functions.

GDPR for AI is the practice of complying with European data protection law when AI systems process personal data.

A golden dataset is a small, high-quality set of human-verified examples used as the ground truth for evaluation.

Grounding is the practice of forcing a model to answer only based on retrieved evidence, not on its training data.

A hallucination is when a model generates something that sounds confident and plausible but is actually false.

A hallucination test is a curated set of prompts designed to expose when a model invents facts.

HIPAA for AI applies the US health privacy framework to AI systems that touch protected health information.

HNSW, or Hierarchical Navigable Small World, is a graph-based index for fast approximate nearest-neighbor search over embeddings.

Hybrid search combines vector similarity search with keyword search to get the best of both.

Inference is the act of running a trained model to produce a prediction or generation for a given input.

Instruction tuning is a training step that teaches a base model to follow human instructions across many tasks.

IVF, or Inverted File Index, partitions embeddings into clusters and searches only the nearest clusters at query time.

A jailbreak is a prompt designed to bypass an AI system's safety training and get it to do something it would normally refuse.

JSON mode is a provider feature that constrains the model to emit valid JSON.

A knowledge graph stores facts as a network of entities and relationships.

LangChain is an open-source framework that provides building blocks for chains, agents, retrievers, and integrations.

LangGraph is a framework for building stateful, multi-step agent workflows as explicit graphs of nodes and edges.

Latency is the time it takes from sending a request to receiving the first or last token of a response.

A latency budget is the maximum end-to-end response time your system is allowed to take.

A license check verifies that model outputs, especially code, do not violate the licensing of training data.

LLM-as-judge is the practice of using a separate language model to score another model's outputs.

LoRA, short for Low-Rank Adaptation, is a lightweight way to fine-tune a model by training small adapter matrices instead of the full weights.

A metadata filter narrows vector search results by structured attributes like author, date, language, or product line.

Mixture of Experts, or MoE, is a model architecture where many specialized sub-networks are trained, but only a few are activated per token.

MLOps is the broader discipline of managing the lifecycle of machine learning models, from training to serving to retirement.

A model card is a short, structured document describing a model's purpose, training data, capabilities, limitations, and risks.

Model Context Protocol, or MCP, is an open standard for how AI clients connect to external tools and data sources.

Distillation is the process of training a smaller model to mimic the outputs of a larger one.

A model gateway is an internal service that sits in front of all model providers, centralizing auth, logging, rate limiting, and routing.

A model router decides which model to send each request to, based on rules, cost, latency, or quality predictions.

Model serving is the infrastructure layer that exposes models via APIs, manages GPUs, and handles batching.

A multi-agent system is an application where several specialized agents collaborate or compete to solve a problem.

Multi-vector retrieval stores several embeddings per document, for example one per paragraph and one per summary.

Observability for AI is the tooling that lets you see what your system is doing, why, and how well.

On-call for AI is the practice of having a rotation responsible for incidents in AI systems, including quality drops, provider outages, and runaway cost.

Output parsing is the step that converts a model's free-text response into a structured data type your code can use.

Output sanitization is the post-processing of model responses to remove or rewrite anything unsafe before it reaches the user.

Parameters are the numeric weights inside a neural network that get learned during training.

Parent document retrieval indexes small chunks for matching but returns larger parent chunks for context.

Perplexity is a metric that measures how surprised a language model is by a given piece of text. Lower is better.

PII detection identifies personally identifiable information like emails, phone numbers, or government IDs in inputs and outputs.

A planner agent is a sub-agent whose job is to break a goal into ordered sub-tasks before any tool is called.

A prompt cache stores model responses for prompts the system has seen before, so repeated calls return instantly and for free.

Prompt chaining is the pattern of breaking a complex task into a sequence of smaller prompts, each feeding the next.

A prompt debugger is a tool that lets you inspect, edit, and re-run prompts with full traces of inputs, outputs, and errors.

Prompt engineering is the practice of crafting model inputs to reliably get the outputs you need.

A prompt firewall is a middleware layer that inspects every prompt for injection attempts, PII, and policy violations.

Prompt injection is an attack where malicious instructions hidden in user input or external data hijack the model's behavior.

A prompt registry is a central store of all production prompts with metadata, owners, and evaluation history.

A prompt template is a reusable string with variables that get filled in at runtime.

Prompt versioning is the practice of treating prompts like code, with explicit versions, change history, and rollback.

QLoRA is LoRA applied on top of a quantized base model, so fine-tuning a large model on consumer GPUs becomes possible.

Quantization compresses a model by storing its weights in fewer bits, for example 4 or 8 instead of 16 or 32.

Query expansion rewrites or augments a user's question with extra terms or paraphrases before retrieval.

Quota management allocates and enforces token, request, or dollar budgets across teams, products, or customers.

RAGAS is an open-source framework for evaluating RAG systems across faithfulness, answer relevance, context precision, and context recall.

Rate limiting caps how many requests or tokens a user, tenant, or service can send in a given window.

ReAct is a prompting pattern where the model alternates between reasoning steps and action steps.

Recursive retrieval traverses a tree of summaries down to leaf chunks based on what looks most relevant.

Red-teaming is the practice of probing an AI system for unsafe, biased, or exploitable behavior before launch.

Regression testing for prompts re-runs your eval set on every change to catch quality drops before they ship.

Reranking is a second-pass scoring step that reorders retrieved passages using a more expensive model for higher precision.

Retrieval is the step in a RAG system that finds the most relevant passages for a given query.

Retrieval-Augmented Generation, or RAG, is a pattern where a model retrieves relevant documents from a knowledge base before answering.

RLHF, or Reinforcement Learning from Human Feedback, is a training step that uses human preference ratings to shape model behavior.

Semantic search finds documents based on meaning rather than exact keyword overlap.

A shadow deployment runs a new model or prompt in parallel with production but does not return its output to users.

SOC 2 for AI applies the standard SOC 2 trust principles to AI-specific risks like prompt injection, model abuse, and data leakage.

A span is a single unit of work within a trace, such as one model call, one retrieval, or one tool invocation.

Sparse retrieval uses high-dimensional, mostly-zero vectors based on keywords to score documents.

Streaming inference returns tokens as they are generated, so users see output incrementally instead of waiting for the full response.

Structured output is the broader feature of forcing model responses to conform to a schema, like JSON, an enum, or a function-call signature.

A sub-agent is a specialized agent that handles one part of a larger task, typically called from a supervisor agent.

A supervisor agent routes incoming requests to the right specialized sub-agent and aggregates results.

A system prompt is a special instruction set at the start of a conversation that tells the model its role, tone, and constraints.

A system prompt pattern is a reusable structure for system prompts, typically covering role, goal, constraints, tools, and format.

Temperature is a number, usually between 0 and 2, that controls how random the model's output is.

A text splitter breaks loaded documents into chunks that respect natural boundaries like headings, sentences, or token counts.

Throughput is how many tokens per second a serving setup can produce across all concurrent requests.

Token accounting tracks input and output token usage per request, per feature, per customer, and per model.

Tokens are the small chunks of text that large language models actually read and write, usually a few characters or part of a word.

Tool use is when a language model decides to call an external function, API, or search instead of answering from its own knowledge.

Top-k sampling restricts the model to choose only from the k most likely next tokens at every step.

Top-p sampling, also called nucleus sampling, limits the model to the smallest set of next-token candidates whose combined probability exceeds p.

Tracing captures the full path of a request through a chain or agent, including every model call and tool invocation.

The transformer is the neural network architecture that powers nearly every modern large language model.

A transparency report is a public document that describes how an AI system was built, evaluated, and is used.

Tree of thought is a prompting pattern where the model explores multiple reasoning branches in parallel before picking the best one.

A vector database stores embeddings and supports fast nearest-neighbor search over them.

Watermarking embeds an invisible signal in AI-generated content so it can be detected later as machine-made.

Zero-shot prompting asks the model to perform a task with only instructions, no worked examples.