In this video we look inside the models used by coding agents. We'll explore how a neural network is built from individual neurons, and see how training changes its weights. Understanding these mechanisms helps explain where a model's abilities come from. We'll go over neural networks here very quickly. It's okay if you don't follow every detail. The subject is taught more at length in later courses on machine learning.
Neural networks were loosely inspired by the brain. Here's a cartoon of a real neuron. It collects signals through its dendrites, on the left.
When the combined signal crosses a threshold, the neuron fires, sending a pulse down its axon to the right.
An artificial neuron uses a simplified version of this idea: combine several inputs, then compute an output.
Let's build one artificial neuron. Its inputs are real numbers, denoted x one, x two, and x three.
We multiply each input by a weight, which represents the strength of its connection to the neuron. We add these weighted inputs together, then add a bias term, b. The result is a single number, z. A weight can be zero when an input makes no contribution. A negative weight means that increasing that input reduces z.
We then pass z through a non-linear activation function. One common choice is ReLU, which returns the larger of z and zero. Negative values become zero, and positive values pass through unchanged. This change in slope makes the function non-linear. Without non-linear activations, stacking these weighted sums would still give us only a linear transformation plus a bias.
The result is the neuron's output, y.
The neuron combines a weighted sum, including the bias, with an activation function. We can connect many of these simple computations to form a neural network. Their combined computation can represent much more complex relationships.
To build the network, we start with a vector of input numbers.
One hidden neuron takes a weighted sum of every input, then applies ReLU, exactly like before.
Putting several neurons side by side gives us a layer. The neurons receive the same inputs, but each has its own weights and bias, so each computes a different function of those inputs.
The outputs of the first layer become inputs to another layer. Stacking layers lets the network build more complex computations from the results of earlier ones.
The final layer produces the network's outputs. We denote this prediction y hat.
Since information flows one way, from left to right, such networks are called feed-forward networks.
Let's use the network for a concrete task: recognizing a handwritten digit. A scanned image is a grid of pixels. Our example shows the digit three on a small grid, five pixels wide and seven pixels high.
We encode each pixel as a number, from zero for black to one for white. The five by seven grid gives us thirty-five numbers, which form the input vector. These numbers describe pixel brightness, but they do not directly tell us which digit is present. We'll train a neural network to recognize the digit from examples, rather than writing the recognition rules ourselves.
We feed the vector through a feed-forward network, just like the one we just discussed.
The network outputs ten numbers, one score for each digit from zero through nine. A higher score means the network favors that digit more. These scores do not have to be probabilities.
We use the digit with the largest score as the network's prediction. Here the score for three is largest, so the predicted digit is three.
We have not written a rule describing the shape of a three. The trained network can recognize it because its weights were adjusted using examples. The recognition rule is represented across those weights, rather than written as a readable sequence of instructions. To understand how the weights acquired those values, we need to look at training.
Instead of writing rules, we gather examples. We need lots of scanned digits.
Each image is paired with its correct label, the known answer. We denote that answer y.
A collection of these image and label pairs is called a training set. More varied, correctly labelled examples give the network more opportunities to learn the patterns it needs.
Training needs a number that measures the prediction error. For one image, the network produces ten scores, shown here as purple bars. Together these scores form the prediction, y hat.
We also write the correct answer as ten numbers: one at the true digit and zero at every other digit. This is the target, y, shown by the orange dots.
We compare each predicted score with its target. For the true digit, three, the prediction is below one. The other scores are above their targets of zero. We square each difference and add the ten results. Squaring makes both positive and negative differences contribute positively, and gives larger errors more weight. The result is the loss for this image. A loss of zero means every output exactly matches its target.
For the whole training set, we add the losses for all the images and divide by the number of examples, N. This gives the average loss. Training adjusts the weights to reduce that average loss and improve the digit predictions.
How do we reduce the loss? With the training set fixed, changing the weights changes the predictions and therefore the loss. We can treat the loss as a function of the weights. To illustrate this, the graph shows the loss as a function of one weight. We initialize weights randomly and start at a point on this curve. The bowl is a simple example of a loss curve, rather than the full shape of a network's training problem.
Ask a simple question about this weight. If I nudge it up or down a little, does the loss grow or shrink? In other words, is the network better at prediction when we nudge the weight up, or is it better when we nudge this weight down?
The slope of the curve is the derivative of the loss with respect to this weight. Its sign tells us which direction increases the loss, and its magnitude tells us how steep the change is.
Once we have the slope, we subtract it from the current weight (after scaling it by some constant factor). This moves the weight toward lower loss for a sufficiently small step. The procedure is called gradient descent. The small number, eta, is the learning rate. It controls how large an update we make at each step. You can work with coding agents without implementing this algorithm. It is a basic method for training neural networks, and a machine learning course would develop these steps in more detail.
We repeat these small updates over many examples. For this digit network, we can go through the training set many times.
To update all the weights, we need the derivative of the loss with respect to each one. An algorithm called backpropagation computes those derivatives efficiently. It makes a forward pass through the network, and then a backward pass.
We'll leave the derivation of backpropagation to a machine learning course. For our purposes, the important sequence is to make a prediction, measure its loss, compute the derivatives, and use them to update the weights.
When we finish training, we keep the learned weights fixed while using the network. Making a prediction does not update them.
To use the network, we run a forward pass on a new input, such as a digit image that was not in the training set. The network produces a prediction. This is called inference.
With fixed weights, this network is a fixed function from input to output. We can evaluate that function repeatedly, using the same weights for each new image. The same distinction applies to a language model: ordinary inference uses fixed weights and does not train them on your conversation. The model can still use new information in its context. A provider can also train and deploy a newer version, but that is separate from the inference taking place when you ask for an answer.
Let's recap. We built a neuron from a weighted sum, a bias, and a non-linear activation function. Connecting neurons into layers gave us a feed-forward network. We then trained that network on labelled examples. The loss measures prediction error, backpropagation computes its derivatives, and gradient descent uses them to update the weights. During inference, we use the learned weights without changing them. Language models use these same basic training ideas in larger and more complex networks. The next video looks at the data and computation needed to train them at that scale.