Concepts

AI winters were periods when inflated expectations collided with limited results, and funding and interest in AI collapsed for years.

Expert systems captured a human specialist's knowledge as hundreds or thousands of if–then rules, and were AI's first big commercial success.

Traditional programs turn rules and inputs into answers; machine learning turns inputs and answers into the rules — learning the knowledge from examples instead of having it written in.

Rule-based AI stores knowledge as if–then rules and derives conclusions by chaining them together, forward from facts or backward from goals.

Search solves a problem by exploring sequences of possible actions from a start state until one reaches the goal — and a good heuristic decides which possibilities to explore first.

Symbolic AI represents knowledge as explicit symbols and rules written by people, and produces intelligent behaviour by manipulating them — through logic and search.

The knowledge-acquisition bottleneck is the discovery that the hardest part of rule-based AI is getting the knowledge in: much of what experts know is tacit, full of exceptions, and too vast to write down.

The Turing test replaces the vague question 'Can machines think?' with a concrete one: can a machine's conversation be told apart from a human's?

Knowledge representation is the problem of writing down what a system knows — objects, categories, relations, defaults — in a form a machine can reason with.

Planning means finding a sequence of actions that turns the current situation into a goal situation, using a model of what each action requires and changes.

Conditional probability asks how likely something is given what you already know, and Bayes' theorem tells you how to flip it — from P(evidence | cause) to P(cause | evidence).

Cross-entropy loss is the negative log of the probability a model assigned to the correct answer — near zero when the model is confidently right, large when it is confidently wrong.

A derivative measures how much a function's output changes when you nudge its input, and the gradient collects those rates for every input at once — pointing in the direction of steepest increase.

The dot product multiplies two vectors entry by entry and adds the results, giving one number that is large when the vectors point the same way.

Entropy measures how uncertain a distribution is — the average surprise of its outcomes — and it sets the lower limit on how compactly you can encode them.

The expected value is the probability-weighted average outcome, and the variance measures how far outcomes typically spread around it.

Gradient descent minimizes a loss by repeatedly nudging every parameter a small step in the direction that decreases the loss fastest — the negative gradient.

KL divergence measures how much one probability distribution differs from another — the extra surprise you pay for using the wrong distribution.

A loss function turns 'how wrong is the model?' into a single number, so that learning becomes the problem of making that number small.

Multiplying a vector by a matrix transforms it — every output number is a dot product of one matrix row with the input — and that is exactly what a neural-network layer does.

Momentum smooths gradient steps by keeping a running average of past gradients, and Adam adds a per-parameter step size based on how large each parameter's gradients have been.

Perplexity is the exponential of the average cross-entropy per token — roughly, the number of options a language model is effectively choosing between at each step.

A probability distribution assigns a likelihood to every possible outcome — and nearly every modern model's output is a distribution rather than a single answer.

The probability of a whole sequence equals the product of each element's probability given everything before it — the identity that turns 'model language' into 'predict the next token'.

Every measured number — an accuracy, a benchmark score, a loss — is computed from a sample, so it carries uncertainty, and a difference smaller than that uncertainty isn't evidence of anything.

Softmax turns any list of scores into a probability distribution — positive numbers that sum to 1 — giving exponentially more weight to larger scores.

Stochastic gradient descent estimates the gradient from a small random batch of examples instead of the whole dataset, trading a little noise for enormous speed.

In deep learning a tensor is just an n-dimensional array of numbers, and keeping track of its shape — batch × sequence × features — is most of the practical work of reading and writing models.

The chain rule says the rate of change through a chain of functions is the product of the rates of change of each link — which is exactly how gradients flow backward through the layers of a network.

A vector is an ordered list of numbers, and in machine learning it is how every object — a house, a word, an image — becomes something a model can compute with.

Data leakage is when training or evaluation data contains information that won't be available when the model is actually used — making results look far better than they really are.

Distribution shift is when the data a model meets in use differs from the data it was trained on, so its measured performance no longer applies.

Accuracy alone can mislead; precision (how many flagged items were right), recall (how many true items were caught) and the confusion matrix show what kind of mistakes a classifier makes.

A supervised learning problem is a table: each row is an example described by features, and the label is what the model must predict — a category (classification) or a number (regression).

The goal of learning is generalization — good performance on data the model has never seen — and a model that memorizes its training data (overfits) or is too simple to capture the pattern (underfits) fails at it.

Classical ML learns only the final mapping from features to labels — people design the features — while deep learning learns the features too, directly from raw data.

Linear regression predicts a number as a weighted sum of the features plus a constant, choosing the weights that minimize the average squared error on the training data.

Logistic regression is a linear classifier: it computes a weighted sum of the features and squashes it through a sigmoid to get a probability, trained by minimizing cross-entropy.

Regularization is anything that discourages a model from fitting the training data too closely — most commonly a penalty on large weights — so that it generalizes better.

Learning paradigms differ in where the training signal comes from: human-provided labels (supervised), structure in the data alone (unsupervised), or labels manufactured from the data itself (self-supervised).

A decision tree predicts by asking a sequence of yes/no questions about the features; a random forest averages many randomized trees to get a much more accurate and stable model.

k-means groups unlabelled points into k clusters by alternating two steps: assign each point to its nearest centre, then move each centre to the mean of its points.

Naive Bayes classifies by applying Bayes' theorem with the simplifying ('naive') assumption that features are independent given the class — crude, but fast and often surprisingly effective for text.

PCA finds the few directions along which data varies the most, so high-dimensional data can be summarized, compressed or plotted with little loss.

A support vector machine chooses the separating boundary with the widest possible margin to the nearest points, and with the kernel trick it can draw curved boundaries by implicitly working in a higher-dimensional feature space.

An activation function is the nonlinearity applied after each neuron's weighted sum; without it, any stack of layers would collapse into a single linear map.

Backpropagation computes how much every weight in a network contributed to the error, by passing the error backward from the output layer by layer using the chain rule.

A computational graph records every elementary operation of a calculation, so that software can compute exact derivatives of the output with respect to every input automatically.

Neural network architectures evolved by building the structure of the data into the network — convolutions for images, recurrence for sequences — until attention offered a more general way to connect everything.

A multilayer perceptron stacks layers of neurons — input, one or more hidden layers, output — so that each layer builds new features out of the previous layer's outputs.

Representation learning means the network learns its own features: each layer transforms the data into a new representation, from simple patterns in early layers to abstract concepts in later ones.

An artificial neuron computes a weighted sum of its inputs, adds a bias, and passes the result through a nonlinear activation function — a dot product plus a bend.

The forward pass is computing a network's output from its input: layer by layer, multiply by weights, add biases, apply activations.

The perceptron (1958) is a single artificial neuron that outputs 1 if a weighted sum of its inputs exceeds a threshold, with a simple rule for learning the weights from mistakes.

In a deep network the gradient reaching early layers is a product of many per-layer factors, so it tends to shrink toward zero or blow up exponentially with depth — making early layers learn far too slowly or unstably.

Batch normalization rescales each layer's activations to zero mean and unit variance using statistics from the current mini-batch, making deep networks train faster and more stably.

Dropout randomly switches off a fraction of neurons at each training step, so the network can't rely on any single unit and must learn redundant, more general features.

Initialization sets the random starting weights at a scale that keeps signals and gradients roughly the same size from layer to layer, so deep networks can start learning at all.

A convolution slides one small grid of weights (a filter) across the input, taking a weighted sum at every position to produce a feature map.

A CNN stacks convolution, nonlinearity and pooling layers so that early layers detect edges and later layers combine them into parts and objects.

Deep reinforcement learning uses neural networks as the policy and value functions, which let RL scale from toy tables to Atari, Go and, eventually, the fine-tuning of language models.

An agent must balance exploiting the best action it knows with exploring actions that might turn out better; too little exploration locks it into a mediocre habit.

A large labelled dataset (ImageNet), GPU training (AlexNet) and residual connections (ResNet) turned CNNs from a niche method into the default for vision between 2009 and 2015.

A digital image is a grid of pixels, each holding a few numbers, so a colour photo is a height × width × 3 tensor.

A Markov decision process formalises an RL problem as states, actions, transitions, rewards and a discount; a policy says what to do, and a value function says how good a situation is under it.

Policy-gradient methods adjust the parameters of a policy directly, making actions that led to higher-than-expected reward more probable and the rest less probable.

Q-learning learns how good each action is in each state by repeatedly nudging its estimate toward the reward received plus the discounted value of the best next action.

Reinforcement learning trains an agent to choose actions that maximise the total reward it collects over time, learning from the consequences of its own behaviour instead of from labelled answers.

Actor–critic methods train two parts together: an actor (the policy) that chooses actions and a critic (a value estimate) that judges them, giving the actor a lower-noise learning signal.

A spectrogram turns a waveform into a time × frequency picture by measuring the frequency content of short, overlapping windows of sound.

Detection finds and boxes every object in an image; segmentation labels every pixel.

Pooling shrinks a feature map by summarising each small block, usually by its maximum, so later layers see a wider area at lower cost.

Speech recognition maps a sequence of audio frames to a sequence of words; the field moved from hand-built pipelines to CTC-trained networks to large encoder–decoder Transformers such as Whisper.

A Vision Transformer cuts an image into 16×16 patches, turns each into a token vector, and runs a standard Transformer over the sequence.

Attention lets a model build each output from a weighted mix of all the inputs, with the weights computed on the fly from how relevant each input is.

An embedding is a learned vector for an item — a word, token, document or image — positioned so that items used in similar ways end up close together.

A language model assigns a probability to each possible next text unit given the units before it.

LSTMs and GRUs add learned gates to a recurrent network so it can keep, discard and update information more deliberately.

An n-gram model predicts the next word by counting what followed the previous n−1 words in a corpus.

A one-hot vector represents a vocabulary item with a 1 in its own position and 0 everywhere else.

An RNN reads a sequence one step at a time, updating a hidden state that carries information from earlier steps.

A sequence-to-sequence model uses an encoder to read one sequence and a decoder to produce another, possibly of a different length.

A language model receives a sequence of discrete text units and must map each unit to a vocabulary ID before any neural computation can begin.

Early encoder-decoder models compressed every detail of the source sequence into one fixed-size vector before decoding.

Word2vec trains compact word vectors with simple local-context prediction tasks rather than a full neural language model.

GloVe learns word vectors from global word co-occurrence statistics, providing another route to distributional geometry.

A neural language model learns word vectors and a probability function together, so similar contexts can support one another.

Neural machine translation trains an encoder and decoder to map a source-language sequence to a target-language sequence.

A causal mask stops each position from attending to later positions, so a model trained to predict the next token can't simply look at it.

The same Transformer block is wired three ways: encoder-only models (BERT) read in both directions to understand text, decoder-only models (GPT) predict the next token to generate it, and encoder–decoder models (T5) map one sequence to another.

The feed-forward sublayer is a small two-layer neural network applied to each token separately, transforming the information that attention has gathered.

Layer normalization rescales each token's vector to zero mean and unit variance (then applies a learned scale and shift), keeping activations in a stable range.

Multi-head attention runs several smaller attention operations in parallel, each with its own learned queries, keys and values, so a layer can track several kinds of relationship at once.

Positional encodings add information about each token's position, because attention on its own treats a sentence as an unordered set.

A residual connection adds a layer's input to its output (x + f(x)), so each layer learns a correction instead of a complete replacement.

Self-attention lets every token in a sequence look at every other token, decide how relevant each one is, and update itself with a weighted mix of what it finds.

A Transformer block is attention followed by a feed-forward network, each wrapped in normalization and a residual connection — and a Transformer is just many identical blocks stacked.

A GPT-style model reads the tokens so far and outputs a probability for every possible next token; generating text means picking one, appending it, and repeating.

A language model outputs a probability for every next token; a decoding strategy decides which one to use, trading predictability against variety.

A pretrained base model continues text; instruction tuning and reinforcement learning from human feedback turn it into an assistant that answers questions and follows requests, which is what ChatGPT made public in 2022.

Between 2018 and 2020 OpenAI kept the same recipe (a decoder-only Transformer trained on next-token prediction) and scaled it about a thousandfold, and the way the models were used changed from fine-tuning to zero-shot to few-shot prompting.

In-context learning is a model performing a new task from instructions or a few examples placed in its prompt, with no change to its weights.

Three numbers describe a language model's scale: how many learned parameters it has, how many tokens it was trained on, and how many tokens it can attend to at once (its context window).

Train one model on a huge amount of unlabelled text first, then adapt it to each task with a small labelled dataset; around 2018 this replaced training every NLP model from scratch.

Pretraining runs next-token prediction over a very large, diverse text corpus, producing a base model that has absorbed patterns of language, facts and reasoning styles from its data.

A language model's loss falls smoothly and predictably, as a power law, as its parameters, training data and compute grow, which turned 'make it bigger' into a plannable engineering decision.

A tokenizer splits text into subword pieces from a fixed vocabulary and maps each piece to an integer ID; the model only ever sees those IDs.

To predict the next token of diverse human text well, a model is pushed to capture whatever regularities produced that text, which is why a simple objective yields broad abilities, and also why those abilities have characteristic limits.

Some abilities appear to jump from near chance to competent as models grow; whether these jumps are real phase changes or artefacts of all-or-nothing metrics is debated.