Image Classification Is a Promise About Generalization

Image classification is a promise that the rule holds on the next image. The split is the only instrument that tests the promise. The overview note listed evaluation as a stage of the pipeline. The split is that stage doing its work.

A 32 by 32 RGB image arrives as 3,072 numbers

A larger image arrives as a bigger slab of the same values. The first job is to turn those numbers into one label from a fixed set. Understanding is not in the slab.

You cannot hand-write the rules for cat or car in pixels. The same object appears across poses, backgrounds, lighting conditions, scales, textures, and partial views. Every condition adds cases, and the rulebook explodes before it covers one class.

You collect examples because you cannot write the rulebook

So you collect examples. Give the model labeled images, pick a model family, and learn from the examples. Then measure on images the model never saw. A model that succeeds only on the images it memorized is a lookup table with good manners.

Every classifier in this course keeps the same contract.

model.train(X_train, y_train)
y_pred = model.predict(X_test)
accuracy = mean(y_pred == y_test)

Train on labeled images, predict on unseen images, score the match. The implementation behind train and predict changes with every model that follows. The contract stays fixed, which is why the crude baseline is still worth writing.

Nearest neighbor is the honest baseline because it makes you define distance

To train, nearest neighbor remembers every image. To predict, it compares the new image against every stored image and copies the closest label. It forces you to define distance before it can answer anything.

L1 adds the absolute pixel differences. L2 squares them, which punishes large disagreements harder. Neither distance understands an object, and both compare numbers.

The first neighbor can be an outlier. k-nearest neighbor asks the top k neighbors to vote, which smooths small islands of bad decisions. Do not trust the first kid who points at an answer. Ask five nearby and take the vote.

Nearest neighbor pays at prediction time. It stores the whole training set and scans it for every query. That cost grows with the dataset. Whatever this baseline already solves is not evidence for the model that replaces it.

Pixel closeness is not semantic closeness

Move a cat a few pixels to the right and the raw vector changes a lot. The picture still means the same thing to you. Change the background behind a car and the model finds a closer match in the wrong class. The colors line up and the label does not.

You stay invariant to pose, lighting, small translation, partial occlusion, and background clutter. Pixel distance treats each of those as a large change. The overview note named that list as the standing pressure on every vision model. Nearest neighbor fails all of it at once.

A vision system must learn those invariances. It cannot assume them. The repair has to come from the representation, because L1 and L2 read only the numbers they are handed.

Tune on the test set and it becomes training data

Train on the training set. Tune k, the distance metric, the learning rate, and regularization on the validation set. Touch the test set once, at the end. Validation is the wall between exploration and final judgment.

A clean split tells you whether the model learned the object, the dataset, or a shortcut. A dirty one tells you nothing and reads the same.

The split leaks in two ways. Near-duplicate images cross from training into validation, and the validation number then reports memory. Repeated peeking at the test set carries the same information across one decision at a time.

Each leak ends in the same place. The test number stays clean and lies. Peek often enough and you optimize the scoreboard instead of the task. The split is the first honest boundary between memory and vision.

The Builder Test

Sort the misclassified images by confidence and start at the top. Inspect those confident mistakes by category, by background, and by distance to their nearest neighbors. Read the confusion matrix and find the class pairs carrying the real ambiguity.

Some of those mistakes are label noise, and a wrong label is not a model failure. Audit the split before you believe any number. Search for near-duplicates crossing training and validation. Check class frequency and photographer bias in the classes you confuse most. If a shortcut explains the confident mistakes, write the test that falsifies it.

What Carries

The visible label is cat, car, truck, or bird. The hidden label is whether the rule survives outside the training set. Nearest neighbor keeps the data and only changes the comparison rule. The next model throws the data away. It keeps parameters instead.