Inference can outspend the training run that produced the model. The training bill closes when the run ends. The serving bill reopens on every call, and it keeps reopening for as long as people use the model.

Models run on physical infrastructure that spends energy, materials, water, land, and attention. Once a model serves billions of tokens, the serving side of that bill passes the training side. Compute is part of the product, so its cost belongs in design review.

Many teams raise environmental impact as an apology after the benchmark lands. By then model size, the serving path, and the growth in demand are already fixed. The question belongs earlier, while the design is still soft. What did this capability cost, and was the cost worth it?

Compute Has Two Lives

Training emissions are easy to narrate because the run is visible. It starts, it ends, and someone can put a number in a paper. Inference is harder to see and easier to leave out. It spends power on every call, and the calls do not stop.

The training number is a fact about the past. The serving number is a forecast about demand, and it moves every time the product gets more users.

After that crossing, an argument about training efficiency is an argument about the smaller half. The footprint becomes a research question and a product question at once. The product side owns the traffic, and traffic is the term that grows.

The Life Cycle Is Wider Than The Run

Life cycle assessment widens the boundary past the training job. Hardware manufacturing, data center energy, cooling, grid carbon intensity, development experiments, inference traffic, and hardware retirement all sit inside it. Leave one out and the total is a guess.

Energy use depends on hardware efficiency, utilization, run duration, power draw, and cooling overhead. Emissions depend on the carbon intensity of the grid that powered the work. The same job on a different grid emits a different amount. Idle capacity spends power while it waits for work that has not arrived.

Embodied carbon counts from manufacturing to disposal, and the manufacturing half is spent before the first token exists. The runs that failed count too. Hyperparameter sweeps and repeated experiments spend real power, and they rarely reach the headline figure.

The parallelism note put its cost under one rule: nothing is free once it has to move. Energy is that rule in another unit. Power moves in and leaves as heat, and cooling overhead pays to remove it.

Report Compute, Energy, Location, Carbon Intensity, And Method

Treat emissions as a product metric. Estimate them before training, record them during the run, and report them after deployment. A team that cannot state its energy and carbon assumptions cannot price the trade it is making.

A usable report names six things: compute, energy, location, carbon intensity, hardware, and the estimation method. Location is there because the grid sets the carbon in every unit of energy. Method is there because all six numbers are estimates. Publish the method beside the total and a reader can check the assumption.

Efficiency Can Raise Total Use

A cheaper token invites more tokens. Efficiency lowers the cost of one call and can raise the total, because demand expands into whatever the price allows. The saving is real only when total work goes down.

The scaling-laws note weighed a marginal parameter against the loss it bought. Price that same marginal gain in energy and some of those trades stop clearing. Traffic decides how often the cost repeats, so the answer moves as the product grows.

Design Choices Move The Footprint

The alternatives are the ordinary ones: smaller models, distillation, retrieval, routing, caching, quantization, batching, and task-specific systems. Each one removes work the product never needed. The greenest token is the one you did not need to generate.

Two of those arrived earlier in this unit. Routing and retrieval come from the selective-architectures note. Waking a few experts per token cuts the compute each token spends. Moving knowledge into an index keeps it out of the weights.

The adaptation note added the rule that fits here: take the smallest rung that clears the reliability bar. Both notes argued for the lower-footprint option before energy entered the argument.

A team that measures picks model size, caching, batching, distillation, and deployment differently from a team that guesses.

The Builder Test

Draw the life cycle of one system you run: training, evaluation, deployment, repeated inference, hardware, and retirement.

• Split the compute into training experiments, failed runs, and serving traffic.
• Mark the stages where you can state a number today, and the stages where you are guessing.
• Find the design change that removes the most repeated work, and price it in energy.

The pile that grows is the one to design against. The stages you are guessing at are the ones to instrument first.

What Carries

Carry the full bill. The greenest system avoided computation nobody needed. Useful work per unit of cost is how that shows up in a review.

Take one deployment claim and trace it from the corpus to the carbon. If it survives the whole trip, the claim is worth defending.

Take the least invasive adaptation that clears the reliability bar. A model can learn the format of helpfulness without learning the work. Heavier methods make that failure easier to buy and harder to see.

Pretraining creates broad capability. Adaptation points that capability at a task, domain, user, or policy. Every method changes one of three things: the model weights, the input, or a small learned module beside it.

Pretraining Makes Breadth, Adaptation Buys A Job

A general model is broad on purpose. Adaptation aims that breadth at one job. The job is a domain, an instruction style, a policy, a tool interface, or a user population. The gain arrives when the target behavior becomes easier for the model than the generic behavior.

Find out what the model already holds before you change anything. Probing trains a small head on frozen representations and reports what the encoding contains. When the capability is already there, steering reaches it and new weights buy nothing.

Prompt, Retrieve, Adapt, Then Fine-Tune

Four rungs, four prices. Prompting is cheap and reversible. Retrieval adds fresh facts and leaves the weights untouched. Adapters add small trainable modules that swap by task or domain. Full fine-tuning updates weights on supervised or preference data and gives the most control at the highest cost.

Between prompting and full weights sit the tuned inputs. Prompt tuning learns soft prompt vectors. Prefix tuning injects learned vectors into attention layers to steer generation, and the base model stays fixed for both.

Which layer holds the missing piece? If the answer is current knowledge, retrieval is safer than new weights. If the answer is format, prompting or adapters do it. The trap is fine-tuning before you prove the problem needs it.

The selective-architectures note moved knowledge into a retrieval index and away from the weights. That is the same selection question in a new place. Routing decides what wakes. Adaptation decides what changes.

Flexibility Trades Against Stability

The trade is flexibility against stability. Full fine-tuning changes the most and risks forgetting and deployment complexity. Lightweight adaptation is cheaper and modular, and it sometimes fails to move behavior far enough.

Fine-tuning does not add knowledge the way a document upload adds a file. The weights move, and behavior on tasks you never adapted moves with them. You pay for depth whether or not you measure the cost.

Modularity is why adapters stay in the running. One base model serves many jobs when small task-specific parameters swap in and out. A second set of full weights is a second artifact to serve, evaluate, and roll back.

A Model Can Learn The Format Of Helpfulness

Preference and instruction tuning teach more than facts and formats. They teach the model which behaviors humans prefer in an interaction. A model that learns the preferred shape of an answer has the costume. Whether it can do the work is a separate measurement.

Adaptation can improve the benchmark and still narrow the model into a brittle local trick. The failing version fine-tunes until the target examples look good. It teaches surface style, benchmark-specific habits, and refusal patterns that break on the first unfamiliar input.

Catching the costume takes examples the tuning never saw. Use held-out tasks, adversarial checks, and rewordings that keep the work while changing the surface. Score the capability, and treat matching phrasing as weak evidence.

The first harms note named the rule: every threshold picks an error. A refusal boundary is a threshold. Tune it on examples that all look alike, and the model refuses the wrong things once real traffic arrives.

Measure The Regression Beside The Target

A serious pass measures the target and the damage in the same run. Skip the damage and you ship an improvement you cannot price.

• Target-task improvement, scored on tasks held out from the tuning data.
• General ability regression on the tasks you never meant to touch.
• Safety regression, including refusals that now fire on ordinary requests.
• Domain shift, measured on traffic collected outside the tuning distribution.
• Format overfitting, caught by rewording the same task and scoring again.
• Interface match between the adaptation data and the deployed contract.

Test outside the tuning set every time, and keep a holdout that represents the deployment world. Define that world before you build it. The capabilities note called it the interaction contract: prompt format, examples, context, decoding rule, tools, retries, refusal policy, and scoring.

Run this pass before launch, while the method is still cheap to change. After launch, the same comparison costs a migration and a second round of evaluation.

The Builder Test

Take one task you already ship. Build the retrieval-prompt version and the adapter version of it. Run both against the same held-out contract.

Record cost, reversibility, evaluation burden, and failure risk for each one. Ship the cheaper rung that clears the bar. Keep both sets of measurements, because the next task starts from them instead of from an argument.

What Carries

Adaptation stays as small as possible and as strong as necessary. Anything more is extra risk and extra maintenance.

Every rung on that ladder costs compute, and the bill continues long after the launch.

Mixture-of-experts wakes a few experts per token. Retrieval moves knowledge out of the weights and into an index. Both buy capacity, and both make selection the new failure point.

Total parameters stop describing the bill. A router decides which parameters wake for this token, and that decision sets the cost.

Conditional Compute Asks Which Parts Must Wake

Dense scaling spends every parameter on every token. Every layer runs and every weight multiplies, whatever the token contains. A selective system asks which parts must wake for this input.

That question buys capacity and specialization. The model holds more parameters than any one token pays for, and different inputs wake different parts. The cost lands on whatever makes the choice.

Reach for a selective architecture when the bottleneck is capacity or knowledge access. A vague product spec does not improve because a router picks the expert. You keep the problem and add a router.

A Router Buys Capacity And Sells You A Routing Problem

A sparse mixture-of-experts model puts a router in front of a set of expert networks. Each token goes to a selected subset, and only that subset runs. Total parameters rise while active compute per token tracks the subset that ran.

Switch Transformer sends each token to one expert. Routing gets simpler and the scaling argument gets simpler with it. One expert per token leaves nothing to dilute a wrong choice.

Load-balancing losses exist because routers collapse. Without pressure to spread load, a router learns to send every token to the same expert. Real traffic pushes the same way, because one domain can dominate a batch. The rest of the capacity sits idle while you pay to hold it in memory.

Training also gets less stable, because a small change in router scores moves a token to a different expert. Specialization is a training outcome, and the balancing term pushes against it. Balance and specialization pull in opposite directions, so you measure both.

Retrieval Moves Knowledge Out Of The Weights

Retrieval attacks a different bottleneck. RAG conditions generation on passages pulled from an index, and RETRO retrieves chunks during generation itself. Both lower the pressure to memorize everything in parameters.

A secret trained into the weights resists deletion, as the security note showed. A row in an index leaves when you delete the row. The legality note ended at a takedown, which is cheaper to serve against an index than against a checkpoint.

Attribution turns from a promise into a pointer to the passage that produced the sentence. Weights are an expensive place to keep a fact that changes, and the index updates without a training run. The retriever now sits on the critical path, and a wrong passage becomes a confident wrong answer.

A retriever can also return nothing useful and leave the model to answer from the weights anyway. That failure reads as a normal answer, and the log is the only place it shows.

Selection Must Be Inspectable

The overview note set one rule. When a model surprises you, name the layer that did the work. A selective system adds a layer that chooses, so the router and the retriever join the weights as suspects.

Log the identity of the chosen path rather than a score for it. Which expert handled this token? Which passage did the retriever return? A system that cannot answer those two questions has added complexity nobody can debug.

Treat the path log as a shipped feature. Without it, a regression in one domain arrives as a support ticket instead of a metric.

Sparse Compute Saves Work Only When Routing Cooperates

Routing overhead has a unit, and the parallelism note named it. Communication competes with compute, and a cluster that waits is not learning. A router adds communication, because tokens must travel to whichever device holds their expert.

Routing overhead, expert imbalance, and brittle specialization each spend part of the saving before it arrives. Memory holds every expert whether it fires or not. Batching gets harder when the tokens in one batch route to different experts.

Fewer active parameters describes one line of the bill. Wall-clock time for a batch covers the rest. Routing adds to that number before conditional compute takes anything off it. A saving you cannot see in the clock is arithmetic about parameters.

The Builder Test

Take one batch of real traffic and log the selected path for every token. Keep the expert identity and the retrieved passage ids next to the output. Then read three numbers off that log.

• Expert load across the batch, which shows whether the router collapsed onto a few experts.
• Quality by domain against a dense baseline, because an average gain hides the slice that got worse.
• Active compute and the communication time the router added, counted together.

Keep the dense baseline alive long enough to compare. A selective system without a baseline reports only its own numbers.

If expert load is lopsided, fix the balancing term before you buy more experts. If one domain lost quality, send that domain to the dense path until the router earns it back.

What Carries

When a model chooses what to use, the choice is part of the model. I treat a router and a retriever as components with owners and dashboards, the same as a service. Without that, the capacity you bought stays theoretical.

Routing decides what wakes. Adaptation decides what changes.

Small probe runs estimate the slope before a frontier run spends real money. The curve they return speaks for the setup that produced it and for nothing else. Carry it past that setup and the number it gives you is a guess.

Scaling Laws Turn Ambition Into Accounting

A scaling law measures how loss changes as you vary model size, dataset size, and compute. Kaplan-style work found smooth power-law improvement across all three. Smoothness makes the small experiment worth running, because a small run forecasts a large one. The goal is to put a training budget where it buys the most capability.

Train several smaller models across a range of sizes and token counts, then fit the trend. The fit turns a future expense into a present decision, and it makes the large run less blind.

I read a pilot as a purchase of information about the slope. A pilot that returns a leaderboard number and no slope buys nothing. Answer the resource-allocation question first, and chase the leaderboard after.

Compute-Optimal Training Balances Parameters Against Tokens

Compute-optimal work sharpened that lesson into an allocation rule. A fixed compute budget splits between parameters and tokens. Spend too much on parameters and too little on data, and the model underlearns. Spend too much data on a small model, and the model cannot use the signal. Find the balance before the bill gets large.

You pay for both failures out of the same fixed budget. Every parameter you add is a token you do not train on.

A token budget is also a step count on a cluster. Nothing is free once it has to move. The parallelism note priced what each step carries: gradients, activations, parameters, and optimizer state. Every extra token buys signal and pays communication.

Diagnose The Bottleneck Before You Buy The Next Parameter

Four diagnoses cover the common cases: parameter-limited, data-limited, compute-limited, and evaluation-limited. A bigger model is a strategy only when the data budget already fits. Otherwise the new parameters sit on top of a corpus that cannot feed them.

Each diagnosis names a different purchase. Parameter-limited buys capacity, and data-limited buys tokens or a cleaner corpus. Compute-limited shrinks the plan until it fits the budget you hold.

The spend has four axes. Fixed compute buys parameters, tokens, data quality, and evaluation. Cleaning the corpus and building the evaluation draw on the same budget as the run.

Evaluation-limited is the case a loss curve cannot see. Loss falls, and the number the product cares about stays flat. Downstream evidence settles it. The capabilities note supplies the form.

Record the interaction contract, then read the failure slice under it. Prompt format, decoding, and scoring all belong in that record. Fix the evaluation first, because a blind measurement leaves every other diagnosis unproven.

The Curve Speaks Only For Its Own Setup

The fit is empirical. Someone measured points and drew a line through them, which is why the measured range matters. Five assumptions sit behind that fit: data quality, model family, optimizer, token budget, and evaluation target. The line holds while all five hold.

Smoothness inside the measured range says nothing about the region past it. Check the fit quality, and check that the planned run lands inside the range you measured.

The data note gives the mechanism behind a data-quality shift. Mixture weights decide what the corpus treats as normal. Re-weight the mixture and the fit now describes a different object. Contamination bends the line from the other side, because the evaluation set leaks into training.

A changed optimizer or a changed architecture voids the fit the same way. The old slope then describes a model you no longer train.

Scaling Scales The Risks Too

The curve forecasts loss and stops there. Memorization, misuse capability, energy use, and deployment impact all grow with the run. Each one needs its own measurement on its own schedule.

Decide who owns each measurement before the cluster starts. The run produces all four whether or not anyone watches.

The same accounting continues after launch. A larger model costs more per token, on every call, for as long as the product runs. The marginal quality gain has to cover latency, money, and environmental cost. Price that gain against the traffic you serve, rather than against the benchmark you published.

The Builder Test

Take one scaling plot you rely on and name the decision it changes. The list is short: model size, token count, compute budget, or accepted risk. A curve that changes none of them is decoration.

Then write the setup beside the plot. Data mixture, model family, optimizer, token budget, evaluation target. If the planned run differs on any line, refit with a probe sweep before you spend.

What Carries

A scaling law is a planning instrument, and it holds for exactly as long as its assumptions hold. The forecast is conditional on a setup you own and must re-check.

A budget argument has a second answer. Stop spending the whole model on every token.

If communication dominates compute, an expensive cluster waits instead of learning. An idle accelerator costs the same per hour as a busy one. The scarce resource picks the parallel strategy.

Frontier Training Is Distributed Systems Engineering

The syllabus page behind this note is mostly a reading list. Read the references and one lesson repeats. Frontier training of a language model is distributed systems engineering.

Large models exceed the memory and throughput of one accelerator. You slice the training job across devices because nothing else fits. The split is mandatory. The communication bill arrives with it.

You do not need a frontier run to meet this constraint. Any job that outgrows one device inherits the same accounting.

Three Splits, Three Bills

The hard part is deciding what each device stores, what each device computes, and when the devices talk. Three splits answer that, and each one moves the pressure to a different place: memory, communication, synchronization, or idle time.

Data parallelism replicates. Each worker takes a different batch and holds a full copy of the model and the optimizer state. The bill is replication in memory plus a gradient collective on every step. It is the simplest split to write and the first to run out of memory.

Tensor parallelism splits the large matrix operations across devices. A giant layer no longer has to fit on one device. It pays in traffic inside the layer, on every forward and backward pass.

Pipeline parallelism assigns layer ranges to stages and streams microbatches through them. A stage at the front finishes early and waits for the stages behind it. That wait is the bubble. Microbatches shrink the bubble and never delete it.

Real systems combine all three. Sharding saves memory because it makes ownership explicit. That explicit ownership is what you audit when the run stalls.

Communication Is Part Of The Computation

Every split creates messages. Activations, gradients, parameters, and optimizer states move between devices while the arithmetic waits for them. The hidden curriculum is communication.

Nothing is free once it has to move. Parameters, activations, gradients, optimizer state, and tokens compete for the same memory and the same bandwidth. A blocking collective sits inside the step time, and the profiler charges it to the step.

Topology decides which of those messages is cheap. A split that ignores the wiring pays for every message at the slowest link.

Pipeline bubbles, slow synchronization, uneven shards, and memory imbalance make a large cluster behave like a much smaller one. Measure the ratio before you buy another rack. If message time per step grows faster than arithmetic time, more hardware makes the ratio worse.

Name The Wall Before You Choose The Tool

Bottlenecks move. You start short on memory, then short on bandwidth, then short on utilization, then short on patience for debugging.

Upstream choices move them. Width and context length decide how much memory one step needs, and the modeling note filed both under representation. A tokenizer that cuts text into smaller pieces spends more positions, and more positions cost memory. The token budget decides how many steps the run must survive, and the scaling-laws note picks that budget.

Activation checkpointing is the worked example of trading one wall for another. You drop the stored activations and recompute them during the backward pass. Memory pressure falls. Compute per step rises. Take the trade when memory is the wall, and refuse it when compute is.

A Week-Long Run Is A Reliability Problem

A week-long run is a reliability problem before it is a training problem. Throughput, device utilization, stragglers, failed workers, synchronization stalls, and numerical instability are first-class signals. Debugging a distributed job needs observability at the cluster level.

One lost worker costs every step since the last checkpoint. Every other device recomputes that stretch, so the whole cluster pays for one failed machine. The checkpoint interval is a budget decision, and the recovery path belongs in the plan.

A straggler never crashes, so nothing raises an alarm. The step ends when the slowest device finishes, and one slow shard sets the pace for the whole run. Numerical instability belongs on that dashboard too. A diverged run wastes as many device-hours as a dead worker.

The Builder Test

Draw the data movement for one training plan. Say where parameters, activations, gradients, and optimizer state live at every step. Mark what is replicated, what is sharded, what is communicated, and when devices wait.

Then hold the drawing against a real trace. Collectives, bubbles, synchronization, imbalance, memory pressure, and topology all appear there. A plan you cannot draw is not a plan yet. The trace names the part you guessed.

Measure the devices doing useful work at the same instant. That count is smaller than the rack count, and it is the one that predicts the finish time.

What Carries

Scaling training means keeping useful work in motion while communication and memory stay under control. Large-model parallelism is resource ownership plus communication scheduling. Once you can keep a cluster busy, the next question is what to buy with it.

The model learns whatever game the objective writes down. A lower training loss can still hide worse behavior on the task your users need.

Architecture is potential. Training turns that potential into a policy over text. The objective decides which policy, and the optimizer decides how cleanly it arrives.

The Objective Is The Curriculum

Training is practice. Every objective hides part of a sequence and pays the model to put it back. The part you hide is the skill you buy.

Four names cover the usual objectives: continuation, reconstruction, denoising, and conditional generation. Each one is a different game, and each one produces a different model.

A decoder-only model hides the future. It predicts the next token from the left context, and continuation becomes the behavior it does best.

Encoder-only training hides tokens in the middle. Masked language modeling scores the reconstruction, which builds the bidirectional representations that understanding tasks use.

Denoising corrupts the input and scores the repair. A model trained that way learns the meaning of the pieces that went missing.

Encoder-decoder training keeps input and output apart. The model reads one sequence and generates another, which is what translation, summarization, and text-to-text tasks need.

Choose The Objective For The Behavior You Need

A generative assistant, a retrieval encoder, and a translation model must not share one training story without a reason. A retrieval system built on a generator asks the model for a behavior it never practiced.

The capabilities note put the interface inside the claim. The objective is that same commitment one layer down. A chat box asks for continuation, and next-token prediction practices exactly that. A search box asks for a representation, and a retrieval encoder practices that instead.

Write the behavior first. Pick the objective that practices it, then pick the architecture that fits the objective.

This breaks when a team copies a popular objective. The model learns the game you wrote down, and only that game gets scored. A wrong objective costs a full run. You find out at the end.

Optimization Writes Behavior Into The Weights

Adam tracks adaptive moments of the gradients. Precision choices and stability tricks decide whether the objective can move through billions of parameters without wasting the run.

Batch size, learning-rate schedule, precision, data order, and checkpointing leave fingerprints on the final behavior. Architecture decides what a model can learn. Training decides what becomes easy.

Data order matters because the learning rate moves. Examples that arrive while the rate is high shape the weights more than later ones.

Two runs with the same architecture and the same objective can land in different places. The optimizer settings are the reason, so they belong in the record.

An Expensive Run Earns Instrumentation Before It Earns Confidence

Training is more than a push of the run button. Evaluate during the run. Regressions and instabilities cost more when you find them late, because the compute is already spent.

Loss curves, validation slices, gradient stability, data mixtures, and hardware utilization are the evidence. Instrumentation buys one decision: stop the run or keep paying for it.

• Loss shows whether the objective moves.
• Validation shows whether the movement generalizes.
• Slices show who the model leaves behind.
• Systems metrics show what the run cost.

A slice is a validation set cut by the group you care about. An average hides the group that got worse.

Finishing proves the scheduler worked. The model still needs evidence. A run can finish and still learn the wrong distribution, waste hardware, or hide data errors.

A checkpoint nobody can rebuild turns into a run you pay for twice. Two fields decide whether it can be rebuilt. The data note put the corpus upstream of the behavior, so the data mixture travels with the weights. The parallelism note holds the hardware trace. Without it, the run cannot be repeated on other machines.

A Lower Loss Can Hide Worse Behavior

For fine-tuning and continued pretraining, log more than final loss. Track calibration, brittleness, memorization risk, instability, and regressions on tasks the base model already handled.

Calibration is whether stated confidence matches the hit rate. Brittleness shows up when a change of wording flips the answer. A run can improve loss and move both of those the wrong way.

Memorization has a mechanism, and the security note names it: duplication. A string the corpus repeats many times is a string the objective pays the model to reproduce.

A lower loss that breaks a critical workflow is an expensive trade you failed to price. Price it before the run. Name the tasks that must not regress and put them in the validation set.

The Builder Test

State the training signal in one sentence. Name what is hidden, what is predicted, what is scored, and what behavior exploits the score.

The last clause carries the risk. A score the model can win without the behavior is a score the model wins without the behavior.

Write the sentence before the run starts. A signal you cannot state plainly is a signal you cannot debug later.

What Carries

Training is where architecture, data, optimizer, and hardware become behavior. The objective is the lesson the model practices millions or billions of times. Scale adds repetitions. A lesson the model never practiced stays unlearned.

Count the accelerators your plan assumes. A training plan that fits one accelerator is not the plan a frontier run uses.

Tokenization is the first modeling choice, and it lands before a single weight moves. Text becomes tokens. Tokens become vectors. Vectors move through the model, and somewhere in that path strings become geometry.

When output looks strange on code, math, rare names, or non-English text, check the tokenizer. Do that before you invent a psychological story about what the model believes. Each of those four cases turns on how the text was cut.

Text Becomes Tokens Before It Becomes Anything Else

The tokenizer cuts every string into pieces before the model sees anything. Split too finely and the model spends its context on the spelling of common words. Split too coarsely and rare forms, morphology, code, and multilingual text get harder. BPE and unigram tokenizers are compromises between those two costs.

Nothing downstream recovers a boundary the tokenizer erased. A rare name that arrives as fragments stays fragments for every layer above it. The pieces are the only view of the string the model gets.

Text the vocabulary does not cover gets cut into more and smaller pieces. The same sentence then spends more of the context window and arrives with less structure. That cost falls unevenly across languages. The first harms note measured it from the other side. It called the gap a quality-of-service harm and named the group the average was carrying.

BPE Merges Frequent Pairs, Unigram Picks Likely Segmentations

BPE starts from small symbols and repeatedly merges the most frequent adjacent pair into a larger token. The merge table records what was common in the corpus it was fit on. Frequent strings become single pieces. Everything else gets assembled out of leftovers.

Unigram tokenization, the SentencePiece style, learns a vocabulary and then picks the segmentation it scores as most likely. BPE builds pieces up from frequency. Unigram chooses among segmentations a learned vocabulary already scores. Both are fit on a corpus, and both freeze one view of what counts as normal text.

Both choices are locked once training starts. The embedding table is indexed by token ID. A new vocabulary sends every ID somewhere else, and the trained table is dead weight. Changing the tokenizer means training again, so price that decision before the run.

Embeddings Make The Workspace, Attention Moves The Context

Embeddings map token IDs into vectors, which gives the model a space to compute in. Similarity, context, and prediction become vector operations. That space is the working memory, and the tokenizer already decided what occupies it.

Recurrent models walk a sequence one step at a time. Transformers process many positions in parallel, and attention carries information between them. Self-attention builds a contextual representation for each token by mixing in the positions it reads. Positional information keeps the order in place, because attention alone does not tell first from last.

Four choices decide what information can move between positions. Attention decides which positions read each other, and depth decides how many times the model edits what they read. Width gives each position more room. Context length sets the reach.

Modeling Choices Become Infrastructure Costs

Attention is why a long context is expensive. Every position that can read every other position needs memory to hold what it reads. That memory grows with the length you allow. The architecture decision arrives later as a training constraint and as a serving bill.

The parallelism note names the walls this creates. Context length and width set the memory wall before the run starts. Once the model no longer fits on one accelerator, the split has to send activations and gradients across the interconnect. Those messages compete with the work itself.

That makes the tokenizer a serving decision as well as a modeling one. Measure pieces per request on your own traffic before you argue about context length.

Check The Representation Before You Blame The Reasoning

The overview note set the rule for this unit. When a model surprises you, name the layer that did the work. The representation is the layer to read first, because everything a reasoning story explains sits on top of it.

Weird spelling, broken code indentation, rare names, and math notation are all shapes a tokenizer problem takes. From the outside each one looks like a failure of understanding. The mechanism is a boundary in the wrong place.

The risk is treating the representation as invisible plumbing. Bad token boundaries, weak embeddings, a short context, and a poor attention pattern all cost the model capacity. They take it before training gets a fair chance. Fix the representation before you spend a run compensating for it.

The Builder Test

Take one sentence your system handles badly. Trace it through the whole path: tokens, IDs, embeddings, attention context, next-token scores. Write down what each step produced, in that order.

Start at the token boundaries, because they are visible and cost nothing to print. Stop at the first step that sounds like magic. That is the step to slow down on.

What Carries

The model edits vectors until the next-token distribution becomes useful. The representation decides what it can edit. Tokenizer, context length, and attention pattern set that limit. All three are chosen before any data moves.

Architecture is potential. The objective decides which potential the model practices.

Public access, permission to copy, permission to train, and permission to deploy are four different questions. A dataset can clear the first and fail the other three. A rights register answers all four, and it has to exist before the first takedown.

Copyright law asks what you can copy, transform, and generate. Privacy law asks what personal information you can collect, process, store, expose, and delete. Where the law runs behind the system, professional codes still bind the builder.

Legal Exposure Accumulates At Every Handoff

The first obligation lands at collection, and every copy, transformation, and filter adds another. Training, serving, logging, and fine-tuning each add one more. So does the customer who uses the output.

That makes law part of system design. A team that cannot explain provenance, licenses, consent, retention, outputs, and user controls cannot price its risk. The obligations exist whether or not anyone wrote them down.

Four Permissions Live Behind One Dataset

Public data carries four permissions, and reading is only the first. Copying it into a corpus, training on that copy, and shipping the result each need their own. Terms of service and dataset licenses decide which of the four you hold.

The answer stays fact-specific. Source, license, jurisdiction, consent, transformation, memorization, output similarity, product use, and user controls all move it. Copyright analysis alone touches copying, fair use, licensing, and derivative works.

Output similarity is a live question because models memorize. The security note showed the mechanism: extraction turns generation into a search for training text. A corpus the model can reproduce is a corpus whose license follows the output.

Privacy Adds Operational Duties

Copyright asks who owns the data. Privacy asks who is in it. Data subjects hold rights over their own records, and users expect deletion to mean deletion.

Privacy analysis touches collection, purpose limitation, retention, access, deletion, and cross-border transfer. Each item names a system that has to change.

Personal information spreads past the training corpus. Logs carry it, and fine-tuning sets carry sensitive text. A retrieval system can hand a document to a user who cannot open it at the source.

A Product Claim Creates Obligations The Model Cannot Satisfy

A model generates legal-looking text. The claim on your website decides what that text becomes: assistance, publication, advice, or automated decision-making. The model behaves the same way in all four cases. The obligations differ in each one.

Each category brings its own machinery. Attribution, user disclosure, review workflows, and appeal mechanisms turn generated text into a claim you can defend. When the model makes a consequential decision, someone owns the audit trail, and the product decides who.

Build The Rights Register Before The First Takedown

A rights register holds one row per data source, and the row carries provenance, permission, use, and remedy. Provenance describes the source, the filter, and the mixture, the three choices the data note used to explain behavior. The permission column names which of the four you hold, and which you assumed. The use column names what the deployed system does with the data.

Remedy is the hardest column to fill with a real mechanism. A removal path touches four places: the training set, the checkpoint, the retrieval index, and the logs. Deleting a row from the training set leaves the checkpoint unchanged. The security note priced that already, because text that entered training does not leave the weights cleanly.

An honest removal path states what it reaches and when. The retrieval index and the logs clear on request. The training set changes at the next build, and the weights change only at the next retrain. Until then the remedy is a filter at serving time, and the register must say so.

The register reads as bureaucracy until the first deletion request, takedown, or procurement review arrives. Courtroom risk is one part of the bill. Product trust, enterprise adoption, and data partnerships turn on the same record. A download is easy to justify, and a lifecycle is the thing you defend.

The Builder Test

Fill one row for your largest data source. Stop at the first column with no answer, because that gap is the finding.

• Source: where the data came from, and who collected it.
• Permission: the license, the consent basis, and the restrictions that came with it.
• Retention rule: how long the raw data, the derived corpus, and the logs stay.
• Removal path: what a deletion request reaches, and what it cannot reach.
• Output-risk check: the test that shows whether the model reproduces the source.
• Responsible owner: the person who answers when a rightsholder writes.

An empty owner column makes the register a document. A named person makes it a control.

What Carries

The model boundary is bigger than the model file. Law decides what the system consumes, what it remembers, what it produces, and what it decides. Each of those four verbs is a place a rightsholder or a regulator can reach you.

Every constraint so far arrived from outside the model: the corpus, the user, the attacker, and the rightsholder. The next four notes open the machine that produced the model.

Extraction attacks turn generation into a search for memorized text. Repeated sampling, targeted prompts, and a search over completions can recover rare sequences the model saw in training. Once a secret is trained in, you cannot cleanly delete it.

That moves the security question off the database and onto the model. Some of the text a model reads is sensitive, copyrighted, private, or recoverable by an attacker who knows how to ask. The weights join the list of places where your secrets live.

A Model Is A Compressed Artifact Of Its Training Data

Memorization risk rises with the copy count. A rare string that appears many times is distinctive enough to stand out and repeated enough to learn. That is the worst pair of properties a secret can have.

The data note in this unit puts the corpus upstream of the behavior, and duplication is one of the ways that shows up here. A crawl collects the same passage from many pages, so the copy count is set before anyone picks a training mix. Deduplication is the control, and it works only while the data is still a file you can edit.

Treat the model as a data-bearing component. Secrets that pass through training data, logs, prompts, retrieval corpora, or fine-tuning sets do not stay where you put them.

Extraction Turns Generation Into Search

An extraction attack needs no new algorithm. The attacker samples the model many times, steers it with targeted prompts, and searches the completions for text that looks memorized. The model does the retrieving.

Membership inference asks a narrower question. It tests whether one specific record was likely in the training data, and it never has to reproduce a line of that record. Membership in a corpus can be the sensitive fact on its own.

Both attacks share one frame. Name what information entered training and how likely extraction is. Then say which prompts or access patterns make extraction easier, and which controls cut the damage. Privacy work starts when those four answers come from your own system.

The Deployment Wrapper Decides Whether A Flaw Becomes A Breach

Memorization is one entrance. Prompt injection, tool misuse, jailbreaks, a compromised model supply chain, poisoned training data, and leakage through logs use the same stack. The wrapper you build around the weights decides how far each of them travels.

Authentication decides who can query the model. Rate limits raise the cost of the repeated sampling that extraction depends on, and audit logs make the attack visible while it runs. Retrieval permissions bound the private context the model can reach at answer time. Incident response sets how fast you cut access after someone reports a leak.

Red-team prompts are how you find out which of those controls holds before an outsider does. Each control is also a threshold, and the first harms note named what a threshold does. It picks which error you eat. A tight rate limit blocks a real user. A loose one lets the sampling run long enough to find something.

Split Instructions From Untrusted Content

Prompt injection turns serious the moment the model reads untrusted text while holding tools or private context. A fetched web page becomes a privileged instruction, because the model has no structural way to tell your intent from the text it fetched.

Split the two channels in a place the system can enforce. Instructions come from you and from your own code. Fetched pages, uploaded files, and tool output stay data the model reads, even when the text inside them sounds like an order.

Then cap the side effects. A demo shows a model that can do anything it is asked. A product gives the model the short list of actions you have priced, so a successful injection reaches less than the demo suggests.

Some Secrets Must Never Enter Training

Controls split by when you can still apply them. Data minimization, deduplication, and differential privacy for sensitive data are training-time choices, unavailable the day after the run finishes. Monitoring, canaries, and strict tool permissions are what you have left.

After the run, what the model holds is fixed. You can filter the output, refuse the prompt, and revoke the key. The string is still in the weights, and no delete you run reaches it. Privacy work that waits for a leak starts after the cheap moment has passed.

The Builder Test

Plant the canary before you need it. Put a unique string into the training data or the retrieval corpus. Pick one that nobody types by accident, and record where you put it. Then attack your own system.

• Sample the model repeatedly with targeted prompts, then search the completions for the canary.
• Repeat the attack at every access level you ship, from anonymous visitor to internal tool.
• Run a membership inference check against one record you know sits in the corpus.
• Write down which log or filter caught the attack, and how long the catch took.

The access level is part of the result. The same prompt at anonymous and at admin returns two different answers, and both belong in the record. I prefer to find my own canary in a completion, before a stranger finds it for me.

What Carries

Training data is an input surface. Generated text is an output surface. Both of them leak, and a control that watches one of them misses the other.

A secret you cannot delete becomes an obligation you cannot satisfy. The legality note takes up the deletion duty and the removal path. Before the next run, ask whoever owns the corpus which strings a removal request must reach, and whether you can reach them.

The dataset is the first behavior specification a model receives. WebText picked its documents by following outbound Reddit links and treating that as a proxy for quality. That choice landed before a single training step ran.

Data work sits upstream of almost every behavior people later attribute to the model. Overrepresent one domain and the model learns that style. Leave duplicated text in and the model can memorize it. A filter that removes some communities harder than others hands the model that skew.

The Model Arrived Through A Supply Chain

Someone crawled the web, someone wrote the filter, and someone set the mixture weights. Someone else chose what to write down about all three, or chose to write down nothing. Those four decisions reach the user before any prompt does.

WebText, OpenWebText, C4, the GPT-3 mixture, and The Pile each encode a different set of collection and filtering choices. Underneath those sits one more choice: whose text becomes training material. The people who wrote that text are not in the room for it.

Every corpus carries duplication, quality variation, demographic gaps, and domain bias into the training run. None of that arrives in the finished weights with a label attached. I do not trust a behavior explanation that stops at the architecture.

Every Corpus Encodes A Proxy For Quality

Nobody reads a web crawl end to end. So each corpus builder picks a proxy and inherits whatever it correlates with.

WebText used outbound Reddit links as its proxy. That proxy measures the taste of the people posting the links, and the corpus takes on that taste. OpenWebText rebuilt a similar corpus in public, so the proxy outlived the team that chose it.

C4 filtered Common Crawl, and a filter is a value-laden transformation. What the filter drops leaves no mark on the model and no mark on the benchmark. Whole languages can leave a corpus that way.

GPT-3 mixed Common Crawl with WebText, books, and Wikipedia at chosen weights. Chosen is the load-bearing word. A source can sit in that mixture louder or quieter than its raw size, and a person picks the number.

The Pile emphasized breadth across domains: code, academic text, web text, and books. Every domain in the mixture is a register the model gets cheaply. Code is the clearest case, because code in pre-training makes code cheap to imitate.

Source, Filter, Mixture, Deduplication

More data is not automatically better. Repeated, low-quality, or misweighted text makes the model worse and the system harder to defend. Four levers decide which of those you get.

• Source decides domain coverage.
• Filtering decides what disappears.
• Mixture weights decide what becomes normal.
• Deduplication decides which text stops shouting.

Each lever costs something in a different part of the system. Duplicates left in the corpus become the memorization the security note describes. Filtering decides which communities thin out, and that thinning is the disparity the first harms note measured. Mixture weights change data quality, and quality bends the curve the scaling-laws note fits.

Pre-training Absorbs The Incentives Before Any Later Tuning

A dataset carries more than text. It tells the model what is common, what is authoritative, what is safe, and what is cheap to imitate. Pre-training absorbs those incentives before any instruction tuning or preference tuning gets a vote.

Web pages, books, code, academic papers, forums, and synthetic examples do different jobs in the finished model. The design question is which behaviors must become easy. The mixture weights are the answer you gave, whether or not you wrote the question down.

Before You Debug Behavior, Inspect The Mixture

The overview note asks you to name the layer that did the work. When the surprise is about what the model knows, imitates, or repeats, the layer is the corpus.

A user asks why the model knew that, imitated that, or leaked that. If you cannot trace the likely data path, you have a governance gap. Prompt changes do not close it.

Datasheets and dataset cards are the minimum operating manual for a corpus that shapes model behavior. A datasheet records sources, collection process, filtering, intended use, limitations, and known risks. Those six fields answer the traceability question one field at a time. Read them back and you can name the likely data path, or see which field is empty.

Data governance for an LLM system means provenance, deduplication, filtering rationale, contamination checks, privacy review, and a refresh strategy. Each one is a document somebody keeps current. Without them, debugging behavior turns into guessing about a corpus nobody can see.

The Builder Test

Pick one behavior your system shows this week. Trace it to a source, a filter, a mixture weight, or a duplicate. Write that trace in one line and keep it with the model.

If you cannot finish the line, you have found the missing document. Go write it before you touch the model.

What Carries

"We trained it on a lot of text" is where most explanations stop. Quantity without provenance is an admission that nobody can name where the behavior came from.

Write the mixture down while you build it, and keep that document next to the weights. A corpus that can be memorized is a corpus that can be extracted.

Every moderation threshold picks which error you prefer to make. Set the dial loose and harmful content reaches the users you meant to protect. Set it tight and the filter blocks harmless speech. A stricter filter does not make a product safer by itself.

A Fluent Model Is A Cheap Content Engine

The same capability helps a student with a draft and writes a customer email. It produces targeted persuasive text at the same price. Every output looks like text.

The harm lives in distribution, intent, and context. Intent sits with the person holding the prompt. The model reads none of it. The same sentence carries different harm in a private draft and in a ranked feed.

Toxicity evaluation probes what a model continues when the prompt pulls toward harmful text. RealToxicityPrompts makes that continuation behavior visible, including completions that turn abusive or identity-targeted.

Disinformation runs on production speed rather than belief. The model produces plausible targeted text cheaply enough that people cannot match the pace. Risk then depends on scale, targeting, cost, believability, and the channel that carries it.

Moderation Is A Workflow Around An Imperfect Detector

One classifier at the door leaves most of the problem unsolved. A team that tunes that classifier and declares the work finished misses context, dialect, satire, quotation, and coordinated abuse.

The working system adds thresholding, context review, escalation, appeal, monitoring, and adversarial testing. Attackers adapt, so policy updates belong inside the loop.

Trace one borderline case through those stages before you argue about the detector. The score decides what the system suspects. The workflow decides whether the call is fair and reversible.

Harmful generation crosses prompt, model, decoding, distribution, moderation, appeal, and monitoring. Every one of those stages holds a control, and moderation is a triage system with human consequences.

Generation policy and moderation policy answer two different questions, and they belong in two documents. One names what the model must not produce. The other names what the platform must not distribute. Both name what needs human review. The system logs the evidence behind each call, because a decision nobody can reconstruct cannot be reviewed.

The Threshold Is The Safety Policy

The system gives a piece of content a risk score. A threshold turns that score into allow, warn, route to review, or block. Move the threshold and you move who gets protection and who gets wrongly stopped.

Warn and route to review sit between allow and block, and both spend reviewer time to buy context. Every point of movement trades one error column against the other.

A false positive blocks harmless speech. A false negative lets harmful content through. The setting that fits depends on the product, the user, the context, and the cost of each mistake.

The detector is a model, so it arrives with the same subgroup problem the first harms note measured. Perspective API scores text for toxicity, and it can encode its own biases and blind spots. The threshold inherits that measurement gap. A dialect that scores high for the wrong reason loses at every setting you pick.

The Hard Cases Are Contextual

The same phrase can be quotation, abuse, documentation, satire, or evidence. A classifier blind to context overblocks some users and underblocks others. Its aggregate score stays healthy through both.

A bad-word list is the cheapest context-blind detector. It misses quotation and satire, and it blocks legitimate speech that uses the same words. The system has to separate quotation from endorsement, and a detector that scores words alone cannot.

The appeal path is where context returns to the decision. An appeal costs the operator reviewer time and a stored record of the evidence. The log has to exist before anyone files one, so the bill starts earlier than the complaint. It costs a policy update when the same call keeps getting reversed.

A missing appeal costs the user the speech, with no route back and no record that the decision happened. The user learns nothing about the rule they crossed, and the operator learns nothing about the rule that misfired. That is the labor and governance bill under every moderation system, and a classifier budget hides it.

A Stricter Filter Can Make The Product Less Safe

Tightening the dial has three prices. It blocks the wrong people, teaches attackers where the boundary sits, and pushes users into worse channels.

A single safety number hides those prices. The score improves while a coded phrase still passes. A coordinated disinformation campaign runs straight through the same filter.

Mitigation sits in more than one place: model training, interface constraints, monitoring, policy, and escalation paths. No threshold carries all five.

The Builder Test

Name both errors before you pick the threshold. Write down who the system harms when it blocks too much. Write down who it harms when it blocks too little. Then measure both on the same traffic sample.

Pull the borderline cases out of the log and read them by hand. Route the ones that turn on context to review, and record what the reviewer decided. Re-test against the adversarial set after every policy update. Record the setting you picked beside both error counts, so the next change has a baseline.

What Carries

A safety setting is a record of which error the team chose to make. The errors stay in the system either way. Moving the dial moves who absorbs them.

Both harms notes point upstream to the same place. Before you tune another threshold, read the corpus that decided what the model finds easy to say.

Representational harm, allocative harm, and quality-of-service harm each need their own evidence. One aggregate score supplies none of the three, and it hides the subgroup it fails.

The capabilities note measured what a model can do. Harm analysis names who pays when that model fails, and what the failure costs them. Capability without harm analysis is power without accounting.

The overview note asked which layer did the work. Harm answers to the same question. It enters through the corpus, model behavior, the interface, and the deployment setting. Name the layer and a bad output turns into a decision someone owns.

Models Inherit The Patterns In Their Documents

A language model inherits the patterns of the societies and documents that produced its data. Those patterns come back four ways: worse performance for some groups, biased associations, stereotyped completions, and hierarchy encoded in language.

The data note supplies the mechanism. Filtering decides which text disappears from the corpus. Mixture weights decide which text becomes normal. A dialect that gets filtered out or under-weighted arrives later as an error rate nobody planned.

Performance disparity is the measurable form. Error rates differ across groups, domains, dialects, languages, and contexts. Social categories are contextual and datasets are imperfect, so one group label covers different people in different settings.

Three Harms, Three Kinds Of Evidence

Split the harm before you measure it. The three kinds behave differently, and each one answers to its own evidence.

• Representational harm changes how people are portrayed. Read completions and review the failures by hand.
• Allocative harm changes who gets resources or opportunities. Compare outcomes by group against base rates.
• Quality-of-service harm changes who receives worse behavior from the same system. Slice the error rate before you report it.

The three do not trade off against each other. A model can portray a group badly while its error rates look even across groups. It can allocate resources evenly and still write demeaning completions. The type you measure decides the failure you can see.

Every threshold picks an error. Tighten it and the system produces false positives. Loosen it and the system produces false negatives. The number decides which group eats which mistake, so name the group before you set the number.

The Aggregate Hides The Subgroup

The same aggregate metric hides different harm profiles, depending on how you slice the population. Aggregate numbers are blunt by construction. They average over the people the system treats worst.

Two versions of a system can report the same overall error rate and earn different verdicts. Slice by dialect and one holds flat, while the other fails on the dialect its corpus carried least. The average passes both. The slice passes one.

Hand review of failures catches what no slice defines. A completion can read as demeaning while the error rate stays flat. The dialect slice returns in the moderation note ahead.

Measurement Decides Which Harms Become Visible

Measurement choices define the groups, the prompts, the labels, the raters, the metrics, and the thresholds. Those choices decide which groups, mistakes, contexts, and trade-offs a benchmark can see. Everything outside that frame stays easy to ignore.

Prompt choice decides which behavior a benchmark ever elicits. A behavior that no prompt triggers reads as absent. The same benchmark can reward surface-level mitigation and leave the deeper behavior intact.

The person who writes the rubric decides what counts as harm. The person outside that rubric pays when the measurement misses.

A harm name gives you a category. The number behind it needs a definition, a dataset, a threshold, and an error analysis. Measurement is the first hard step.

The shortcut names harm categories and never tests whether the measurement can see them. That buys moral clarity on the page and operational blindness in the system.

You can act on a harm only after a measurement choice makes it visible.

Start The Harm Analysis Before Launch

A fairness dashboard after launch reports what already shipped. Run the harm analysis on the use case first, and write down four things.

• Name who the data represents, and who it leaves thin.
• Name who a wrong output lands on.
• Write down who can appeal, and the path that appeal takes.
• Price the false positive and the false negative, group by group.

The appeal path is where a person the measurement missed corrects the record. A system without one makes every wrong output final.

Draw the system boundary wide. The model, the prompt, the product interface, and the deployment setting each create harm on their own. The interface is part of the claim, so an analysis that stops at the weights stops early.

Early has a test. The analysis must land while product decisions still change. After launch it becomes a report.

The Builder Test

Take one aggregate metric from a system you run this week. Slice it by group or by dialect, and name the subgroup the average was carrying. Then name the error that costs that subgroup more, and the signal that tells you it happened.

If the slice does not exist yet, that absence is the finding. Build the slice before the next release.

What Carries

A harm definition that cannot name who is affected is too abstract to act on. Carry the affected user into every measurement decision, and the definition stays usable.

Bias in what a model reflects is one problem. What it produces on demand is the next one.

A capability belongs to the model and the interface together. The same checkpoint looks weak under a bare prompt and strong under a benchmark-friendly one. The claim stays incomplete until it names the prompt, the context, the decoding rule, the benchmark, and the failure examples.

A Capability Appears Through An Interface

Language models handle language modeling, question answering, translation, arithmetic, article generation, and grammar correction. They also handle tasks nobody trained them for. Which of those you see depends on how you ask.

Capability claims disagree because the setups disagree. One evaluation sends a bare prompt. Another gives examples, tool access, retrieval, scratch space, and repeated attempts. The checkpoint stays the same across all of them. The interface decides what it can express.

One sweep settles which pair a claim describes. Compare bare prompting, few-shot prompting, tool access, retrieval, scratch space, and repeated attempts on one task. A fragile model scores well under one of those and drops under the rest. If the score swings across that sweep, the claim belongs to the model-interface pair rather than to the checkpoint.

Few-Shot Prompting Writes A Temporary Task Specification

The GPT-3 era made this visible. Engineers scaled a next-token model and few-shot behavior came out of it. Place a few examples in the context window, ask for a continuation, and the model often infers the task format.

Show three examples of country to capital and the model continues the pattern. Phrase the same game badly and it misses. The examples tell the model which game you play. The checkpoint holds no module for that game.

Those examples write a temporary task specification inside the context window. Change the examples and you change the task. Pattern completion behaves like task learning here, and the behavior leaves when the prompt does.

A Benchmark Is A Contract

Benchmarks such as LAMBADA, HellaSwag, TriviaQA, WebQuestions, and arithmetic probes slice capability into observable tasks. Each one declares what counts as input, output, success, and failure. That declaration is the contract the benchmark offers, and it covers only what it names.

Language modeling benchmarks measure predictive fit, usually with perplexity or likelihood. Question answering measures whether the model connects prompt context with stored or retrieved knowledge. Translation needs mapping between languages. Arithmetic needs symbolic precision at every step. A perplexity win and a question answering win are two different claims about one checkpoint.

Scoring moves the number as much as the prompt does. An exact match, a judged answer, and a human rating measure three different things. Record which of the three produced the number you quote. Then check whether the benchmark scores the interface you ship. If it scores a different one, the number belongs to a different product.

Record The Whole Interaction Contract

The benchmark contract covers the task. The interaction contract covers everything around it. The common product mistake is to compare models on one task with no fixed interface. That comparison ranks prompts and publishes the ranking as a model verdict.

The interaction contract has eight fields: prompt format, examples, context, decoding, tools, retries, refusal policy, and scoring. User expectation belongs beside them, because the same output passes one reader and fails another. Write all of it next to the number. Skip it and the number measures an undocumented system.

Decoding belongs in the record even when you never change it. The introduction note set up the decoding fork, and that fork sits inside every capability number you report. Change the decoding rule and the score moves while the weights stay fixed.

The Average Hides The Slice That Fails

Treat novel task behavior as evidence to investigate. The evidence that settles it lives in the failures, so pull the wrong answers and read them. Name what they share, then check whether the shared property appears in the work you plan to ship.

Emergent behavior means a capability became visible under a certain scale and interface. Stability is the open question after that. A model that answers the benchmark and breaks in a messy user workflow gives you a number with no coverage.

Say a model can do a task without saying under what conditions and the measurement turns into a myth. The conditions are cheap to write down. The myth is expensive to unwind after a launch.

The Builder Test

Write the interaction contract for one task you own, every field, in one file. Hand the file to another engineer and ask for the number. If the number they get does not match yours, a field is missing from the file. A contract nobody else can run is a note to yourself.

What Carries

A capability is never separate from the way you ask for it and the way you score it. An interface elicits it, and that interface must travel with the claim. Name the interface and a surprising result turns into an engineering decision.

You can now say what a model does under a named interface. The next question is who pays when that interface fails.

A language model assigns probabilities to strings. Temperature and top-k then decide whether those probabilities read as deterministic autocomplete or as a stochastic writer. The weights do not move between the two.

That definition sounds too small to explain GPT-style behavior. The surprise is how much world structure fits inside one plain objective.

A Language Model Assigns Probabilities To Strings

A next-token distribution is a compressed map of syntax, facts, styles, tasks, and patterns in the training data. Autoregressive language modeling decomposes a sequence into conditional probabilities, one per position. Predict the next token from the tokens before it, then do it again for the position after that.

Multiply or sum those probabilities across a sequence and the model scores the text. A sequence score lets you compare two candidate strings under one model. Scoring and writing run on the same conditionals.

The corpus decides the shape of that distribution. Frequency in the documents becomes fluency in the output. What the documents never carried is what the model finds hard to say. The data note in this unit opens that question.

Generation Is A Loop That Eats Its Own Output

Generation runs a loop. Condition on the prefix, choose or sample a token, append it, and repeat. The model turns its own output back into its own input at every step.

Each appended token joins the next prefix, so a choice made early conditions everything the model writes after it. One low-probability token sampled early becomes a premise the rest of the generation conditions on.

The model assigns scores, normalization turns them into probabilities, and a sampler or a search rule chooses what appears. Change any one of those pieces and the same weights behave differently.

Temperature and top-k are load-bearing settings. A small change in either one moves the visible personality of the system. Decoding is one field of the interaction contract that the capabilities note in this unit asks you to record.

Sparse Counts Cannot Generalize, Learned Representations Can

Shannon, entropy, n-grams, smoothing, and the early neural language models all wrestled with uncertainty over text. Modern language models inherited that problem and changed the scale, the representation, and the interface.

N-gram models expose the old bottleneck. Counts are sparse, so a table of exact strings generalizes badly to a context it never saw. Neural models replace that table with learned representations. Similar contexts then share statistical strength, so an unseen context gets a probability from its neighbors. Scale changed the representation and left the prediction problem where it was.

Every Visible Token Hides Three Questions

Every visible token hides three questions, and asking them blocks a mystical reading of the output.

• Distribution: which patterns the model learned from its training data
• Context: what information is visible in this prompt right now
• Decoding: how the system converts probabilities into text

Probability is the clean abstraction and generation is a concrete process. The first question belongs to the model, and the second to the prompt. The third belongs to whoever configured the sampler, and that is usually you. Two of the three are yours to set before you send the prompt.

Debug The Sampler Before You Blame The Model

Treating output text as if it came from a mind with stable beliefs is the practical mistake. It came from a distribution under a decoding rule, and that is what makes it debuggable.

The easy stopping point is the most fluent sample, read as a belief the model holds. Another rule on the same distribution returns a different continuation, and the uncertainty the fluent sample hid becomes visible.

If the answer is too bland, inspect decoding. If it is confidently wrong, inspect retrieval and the evidence in the prompt. Instability across reruns is a different signal, and it points at prompt sensitivity and sampling.

The overview note asked you to name the layer that did the work. Here the layers are the model, the context, and the sampler. Start with the sampler. It is the cheapest of the three to change.

The Builder Test

Take one prompt and run it under three decoding settings. Read the three outputs side by side and compare uncertainty, diversity, repetition, and hallucination risk. If all three come back nearly identical, the prompt is doing the constraining and the sampler has little room.

Then hold decoding fixed, change one thing in the context, and run it again. That pass measures sensitivity to context. The first pass prices the sampler and the second prices the prompt. Record both settings beside the output, or you keep three samples and no explanation.

What Carries

Text generation is probabilistic conditioning plus a decoding rule. The final text is a system output rather than a property of the weights. Each layer can be tested on its own. A claim about the model that never names the context or the sampler is a claim about one draw.

Keep the weights fixed and change the interface around them, and the same distribution reads as a different model.

A language-model claim has six parts, and a benchmark score is one of them. A claim that reports only the score is unfinished.

The other five sit in layers the benchmark never reaches, and each of them changes what the number means.

A Model Is A Corpus, An Objective, And A Deployment Surface

Count the parameters and you describe one part of the object. The same model is a training corpus, a tokenizer, an objective, and a compute plan. It is also a deployment surface and a set of institutional consequences.

Each of those parts gets a note of its own later in this unit. The corpus belongs to the data note, the tokenizer to the modeling note, the objective to the training note. Compute lands in the parallelism and scaling notes, and the consequences land in the harms, security, and legality notes.

The weights are the cheapest part of the object to describe. They predict the least about what the system does once it reaches users. A strong benchmark score sits next to weak data governance, an unsafe deployment, a privacy leak, or a serving bill nobody checked.

The Sequence Runs Behavior, Responsibility, Machine Room, Frontier

The first two notes ask what a language model is and what it does. Next-token prediction is a small objective, and the behavior it produces does not stay small once the training run gets large. Both notes treat the model as a probability machine that became a general interface.

The middle notes turn to the world around the model. Bias, toxicity, disinformation, data provenance, security, privacy, and law arrive here. Each one asks who benefits and who carries the risk when the model is wrong.

The back half opens the machine. Tokenization and Transformers set the shape of the model, and training objectives and optimization set what it practices. Parallelism and scaling laws turn frontier training into an infrastructure problem with a bill attached.

The last stretch covers scale, sparsity, adaptation, and cost. Selective architectures and adaptation make a system more useful without a bigger dense model. The environmental note prices what the whole plan consumes.

Read The Stack Sideways

The stack reads better sideways than top to bottom. A choice made in one layer arrives as a bill in another, and the other layer usually belongs to a different note.

The tokenizer is a modeling decision. Split text into finer pieces and every sequence gets longer. Longer sequences cost memory, and the parallelism note plans around that memory.

A corpus filter is a data decision. The filter decides which text disappears, and what disappears returns later as the subgroup the first harms note finds inside an average.

Give a model tool access and you make a deployment decision. The model then reads untrusted text while it holds tools and private context, which the security note treats as prompt injection. The duty to disclose what leaked belongs to the legality note.

Whenever A Model Surprises You, Name The Layer

Name the layer and the surprise turns into work you can price. Once the layer has a name, you can inspect it and change it. I keep this rule because a layer with a name has an owner, and you can ask an owner for a number.

The capabilities note opens on one of these surprises. You put examples in the prompt, the model follows the pattern, and the result looks like a new task learned on the spot.

Two layers produced that. Scale is one, because the capability appears only after the model gets big enough. The interface is the other, because the examples inside the context window write a temporary task specification.

Change the prompt format and the capability moves while the weights stay the same. One of those two layers costs a training run to change. The other costs a prompt edit. Name the layer and you know which bill you are about to pay.

A Finished Claim Names Six Things

A behavior starts in data and objective choices. It appears through an interface, an evaluation judges it, and then it reaches real users under real constraints. Each step in that chain leaves something the claim has to name.

• The dataset says where the behavior came from.
• The interface records how you asked, because a different prompt is a different claim.
• The measurement fixes what counted as success and what counted as failure.
• The user is the person who meets the output, under whatever pressure they already carry.
• The failure mode names what goes wrong and on which slice of the traffic.
• The cost of an error is what the wrong answer takes from the person who receives it.

A benchmark score stays true while five of the six stay missing, which makes it easy to report and hard to act on.

The Builder Test

Take one deployment claim you already make about a system you run. Trace it from the source data through the objective and the training run.

Continue through the evaluation contract and the user interface, then the monitoring loop. Stop at the rule that changes a launch decision, and write that rule down.

If the interface you measured under has no name in your notes, stop and write it down. The trace fails at the first step where you cannot name an owner. That step is the layer to fix first.

What Carries

You cannot separate model quality from the stack that made it, serves it, and absorbs its mistakes. Read each layer as its own discipline and every risk looks smaller than it is, because the hardest failures cross boundaries.

The first layer to open is the one every other layer conditions on. Read the next-token distribution first, because every layer above it inherits its shape.