Reference

AI Glossary

The vocabulary of modern AI — context windows, RAG, agents, alignment — defined in plain language.

Plain-language definitions for the vocabulary of modern AI. Every entry is linkable — click a term, share the URL. Machine-readable: glossary.json.

Agent: A software system that uses an AI model to take actions on its own toward a goal, such as browsing the web, running code, or using tools, rather than just answering a single question. Agents typically work in a loop: decide, act, observe the result, and try again.
AGI (Artificial General Intelligence): A hypothetical AI that could understand and perform any intellectual task a human can, across many domains rather than one narrow specialty. There is no agreed-upon test for when AGI would be reached, and experts disagree on whether or when it will happen.
Alignment: The effort to make AI systems pursue the goals and values their designers and users actually intend, and to behave safely and honestly. Misalignment is when a system optimizes for something subtly different from what people want.
ASI (Artificial Superintelligence): A hypothetical AI that would be far more capable than the best humans at essentially everything. It is a step beyond AGI and remains speculative.
Attention: A mechanism inside modern AI models that lets them weigh which earlier words or pieces of input matter most when producing the next piece of output. It is the core idea behind the transformer, the architecture used by most large language models.
Backpropagation: The core training algorithm for neural networks. It works out how much each internal setting contributed to the model's errors and nudges all of them in the direction that reduces those errors, repeated over huge amounts of data.
Benchmark: A standardized test used to measure and compare how well AI models perform on a specific kind of task, such as math, coding, or general knowledge. Benchmarks help track progress, but a high score does not always translate to real-world usefulness.
Chain of thought: A technique where a model works through a problem step by step in writing before giving its final answer, much like showing your work in math. It often improves accuracy on reasoning-heavy tasks.
Chatbot: A program you interact with by typing or speaking in everyday language, and that responds conversationally. Modern chatbots like ChatGPT and Claude are powered by large language models.
Compute: The raw computing power, usually measured as processor time on chips like GPUs, needed to train or run an AI model. More compute generally allows bigger models and more training, and it is one of the main costs and bottlenecks in AI.
Context window: The amount of text, measured in tokens, that a model can take in and consider at once, including both your input and its own response. Anything beyond that limit is dropped, so a larger context window lets a model work with longer documents or conversations.
Deep learning: A type of machine learning that uses neural networks with many layers to learn patterns directly from large amounts of data. It powers most modern AI, including image recognition and large language models.
Diffusion model: A type of AI model that generates images or other data by starting with random noise and gradually refining it into a coherent result. It is the technology behind many popular image generators.
Distillation: A technique for training a smaller, cheaper student model to imitate a larger, more capable teacher model. The goal is to keep much of the quality while cutting the cost and speed of running it.
Embedding: A way of representing words, images, or other data as a list of numbers so that items with similar meaning end up close together. Embeddings let software measure similarity and are a building block of search and recommendation systems.
Emergent abilities: Skills that appear in large models but were not present in smaller ones, seeming to switch on once a model reaches a certain size or amount of training. Researchers debate how real and how predictable these jumps actually are.
Few-shot learning: Giving a model a handful of examples of a task inside the prompt so it can follow the pattern, without any additional training. It contrasts with zero-shot, where no examples are provided.
Fine-tuning: Taking an already-trained model and training it further on a narrower set of examples so it does better at a specific task or adopts a particular style. It is usually far cheaper than training a model from scratch.
Foundation model: A large model trained on broad data that can be adapted to many different tasks, serving as a base that others build on. Large language models are the best-known example.
Frontier model: One of the most capable AI models available at a given time, typically from a leading lab and trained at very large scale. The term comes up often in discussions of AI safety and regulation.
Generative AI: AI that creates new content, such as text, images, audio, video, or code, rather than just classifying or scoring existing data. Chatbots and image generators are common examples.
GPU: Short for graphics processing unit, a type of chip originally built for rendering graphics that turns out to be very good at the parallel math AI models need. GPUs are the main hardware used to train and run modern AI.
Guardrails: Rules, filters, or added safety layers that keep an AI system from producing harmful or off-limits outputs. They sit around the model rather than being part of how it fundamentally works.
Hallucination: When an AI model states something false or made-up as if it were true, often fluently and confidently. It happens because models predict plausible-sounding text rather than looking up verified facts.
Inference: The act of running a trained model to get an answer, as opposed to training it. Every time you send a prompt and get a response, that is inference.
Jailbreak: A prompt or trick designed to get an AI model to bypass its safety rules and produce content it is meant to refuse. Labs continually patch known jailbreaks as they are discovered.
Knowledge cutoff: The date after which a model has no built-in knowledge, because its training data only goes up to that point. Events after the cutoff are unknown to the model unless it is given that information or can look it up.
Large language model (LLM): An AI model trained on huge amounts of text to predict and generate language, which lets it answer questions, write, summarize, and more. Often abbreviated LLM, these models power today's leading chatbots.
Latency: The delay between sending a request to an AI model and getting a response back. Lower latency means the system feels faster and more responsive.
Machine learning: A branch of AI where systems learn patterns from data and improve with experience, instead of being explicitly programmed with rules for every case. Deep learning is one powerful kind of machine learning.
MCP (Model Context Protocol): An open standard, introduced by Anthropic in late 2024, for connecting AI models to outside tools and data sources in a consistent way. It lets developers plug an assistant into things like files, databases, and apps without custom wiring for each one.
Mixture of experts: A model design that splits the network into many specialized sub-models, or experts, and for each input uses only a few of them. This keeps a model large in total knowledge while making each response cheaper to compute.
Multimodal: Describes an AI model that can handle more than one type of input or output, such as text, images, audio, and video, rather than text alone. Most leading models today are multimodal.
Neural network: A computing system loosely inspired by the brain, made of layers of simple connected units whose numeric settings are tuned during training. It is the basic structure underneath deep learning and modern AI.
Open weights: When a model's trained parameters are released publicly so anyone can download, run, and adapt it. This differs from fully open source, since the training data and code may not be shared, and from closed models offered only through an online service.
Overfitting: When a model learns its training examples too closely, including their quirks and noise, and as a result performs worse on new, unseen data. Avoiding it is a central concern in machine learning.
Parameter: One of the internal numeric values a model adjusts during training; together they store what the model has learned. Modern large models have billions or more, and the count is a rough, imperfect indicator of capacity.
Pre-training: The first and largest training stage, where a model learns general patterns from a huge, broad dataset. For language models this usually means learning to predict the next token across vast amounts of text, with later stages like fine-tuning specializing it.
Prompt: The input or instruction you give an AI model to get a response, such as a question, a request, or any text you type in. The wording of a prompt can strongly affect the quality of the answer.
Prompt engineering: The practice of carefully wording and structuring prompts to get better, more reliable results from an AI model. It can include giving examples, setting a role, or breaking a task into steps.
Prompt injection: An attack where hidden or malicious instructions, often buried in a web page or document the AI reads, trick the model into ignoring its real instructions. It is a major security concern for AI agents that browse or process outside content.
Quantization: A technique that shrinks a model by storing its numbers with less precision, so it uses less memory and runs faster. It usually costs a little accuracy in exchange for being cheaper to run.
RAG (Retrieval-Augmented Generation): A method where the system first looks up relevant information from an outside source, such as a document collection, and feeds it to the model so answers are grounded in that material. It helps reduce hallucination and lets a model use up-to-date or private data.
Reasoning model: A model trained to spend extra effort thinking through a problem step by step before answering, which improves performance on hard math, coding, and logic tasks. OpenAI's o1 and DeepSeek-R1 are well-known examples.
Red teaming: Deliberately probing an AI system to find its weaknesses, harmful outputs, and ways it can be misused, so they can be fixed before release. The idea and name are borrowed from security testing.
Reinforcement learning: A training approach where a system learns by trial and error, receiving rewards for good outcomes and penalties for bad ones, and adjusting to earn more reward. It is used both in game-playing AI and in fine-tuning language models.
RLHF (Reinforcement Learning from Human Feedback): A training method where people rate or rank a model's responses, and those judgments are used to steer the model toward more helpful and appropriate answers. It was central to making chatbots like ChatGPT feel useful and polite.
Scaling laws: Observed, fairly predictable relationships showing that model performance tends to improve as you increase model size, training data, and compute. They have guided the industry's push toward ever-larger models.
Speculative decoding: A speed-up technique where a small, fast model drafts several tokens ahead and a larger model checks them in one pass, accepting the ones it agrees with. It produces the same output as the big model alone but faster.
Synthetic data: Training data generated by a computer or another AI model, rather than collected from the real world. It can fill gaps where real data is scarce, expensive, or sensitive, though it carries risks if the generated data is flawed.
System prompt: A behind-the-scenes instruction that sets an AI assistant's role, tone, and rules before the conversation with the user begins. Users usually do not see it, but it shapes how the assistant responds.
Temperature: A setting that controls how random or predictable a model's output is. Lower values make responses more focused and repeatable; higher values make them more varied and creative.
Test-time compute: The idea of letting a model use more computation while answering, for example by reasoning longer or trying multiple approaches, to get better results. Reasoning models rely heavily on this.
Token: A chunk of text, often a word or part of a word, that a language model reads and generates one at a time. Models measure input length, output length, and pricing in tokens.
Tool use: When an AI model calls external tools, such as a calculator, a search engine, or a piece of software, to do things it cannot do reliably on its own. It lets models fetch fresh information and take real actions.
Training data: The collection of examples a model learns from during training. Its size, quality, and biases strongly shape what the model knows and how it behaves.
Training run: A single, complete session of training a model from start to finish on a chosen dataset and setup. For large models, one run can take weeks and cost millions of dollars.
Transformer: The neural network architecture, introduced in 2017, that underlies most modern language models. Its key innovation, the attention mechanism, lets it handle long stretches of text efficiently and learn relationships between distant words.
Turing test: A thought experiment proposed by Alan Turing in 1950, in which a machine passes if a human judge chatting with it cannot reliably tell it apart from a person. It shaped decades of debate about machine intelligence, though many now see it as a limited measure.
Vector database: A specialized database that stores embeddings and can quickly find the items most similar in meaning to a query. It is a common building block for search and for retrieval-augmented generation.
Vibe coding: An informal style of programming, popularized in 2025, where a person describes what they want in plain language and lets an AI generate the code, guiding it by feel and results rather than writing much code by hand. The term was coined by AI researcher Andrej Karpathy.
World model: An AI system's internal representation of how the world works, which it can use to predict what happens next or plan ahead. How much today's models truly have one is an open research question.
Zero-shot learning: When a model performs a task it was given no examples for, relying only on its general training and the instructions in the prompt. It contrasts with few-shot, where a few examples are provided.